-- Logs begin at Thu 2022-09-15 11:45:00 UTC, end at Thu 2022-09-15 12:53:03 UTC. -- Sep 15 12:09:39.339828 dv-cinder-850836-7-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Sep 15 12:09:39.347065 dv-cinder-850836-7-iscsi devstack@keystone.service[81435]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Sep 15 12:09:39.350099 dv-cinder-850836-7-iscsi devstack@keystone.service[81435]: *** Starting uWSGI 2.0.18-debian (64bit) on [Thu Sep 15 12:09:39 2022] *** Sep 15 12:09:39.350099 dv-cinder-850836-7-iscsi devstack@keystone.service[81435]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Sep 15 12:09:39.350099 dv-cinder-850836-7-iscsi devstack@keystone.service[81435]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Sep 15 12:09:39.350099 dv-cinder-850836-7-iscsi devstack@keystone.service[81435]: nodename: dv-cinder-850836-7-iscsi Sep 15 12:09:39.350099 dv-cinder-850836-7-iscsi devstack@keystone.service[81435]: machine: x86_64 Sep 15 12:09:39.350099 dv-cinder-850836-7-iscsi devstack@keystone.service[81435]: clock source: unix Sep 15 12:09:39.350099 dv-cinder-850836-7-iscsi devstack@keystone.service[81435]: pcre jit disabled Sep 15 12:09:39.350099 dv-cinder-850836-7-iscsi devstack@keystone.service[81435]: detected number of CPU cores: 4 Sep 15 12:09:39.350099 dv-cinder-850836-7-iscsi devstack@keystone.service[81435]: current working directory: / Sep 15 12:09:39.350099 dv-cinder-850836-7-iscsi devstack@keystone.service[81435]: detected binary path: /usr/bin/uwsgi-core Sep 15 12:09:39.350640 dv-cinder-850836-7-iscsi devstack@keystone.service[81435]: your processes number limit is 47680 Sep 15 12:09:39.350640 dv-cinder-850836-7-iscsi devstack@keystone.service[81435]: your memory page size is 4096 bytes Sep 15 12:09:39.350640 dv-cinder-850836-7-iscsi devstack@keystone.service[81435]: detected max file descriptor number: 2048 Sep 15 12:09:39.350640 dv-cinder-850836-7-iscsi devstack@keystone.service[81435]: lock engine: pthread robust mutexes Sep 15 12:09:39.350640 dv-cinder-850836-7-iscsi devstack@keystone.service[81435]: thunder lock: enabled Sep 15 12:09:39.350640 dv-cinder-850836-7-iscsi devstack@keystone.service[81435]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Sep 15 12:09:39.350640 dv-cinder-850836-7-iscsi devstack@keystone.service[81435]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Sep 15 12:09:39.389687 dv-cinder-850836-7-iscsi devstack@keystone.service[81435]: Python main interpreter initialized at 0x55d1de5849c0 Sep 15 12:09:39.389687 dv-cinder-850836-7-iscsi devstack@keystone.service[81435]: python threads support enabled Sep 15 12:09:39.389687 dv-cinder-850836-7-iscsi devstack@keystone.service[81435]: your server socket listen backlog is limited to 100 connections Sep 15 12:09:39.389687 dv-cinder-850836-7-iscsi devstack@keystone.service[81435]: your mercy for graceful operations on workers is 90 seconds Sep 15 12:09:39.390092 dv-cinder-850836-7-iscsi devstack@keystone.service[81435]: mapped 403077 bytes (393 KB) for 2 cores Sep 15 12:09:39.390173 dv-cinder-850836-7-iscsi devstack@keystone.service[81435]: *** Operational MODE: preforking *** Sep 15 12:09:39.390380 dv-cinder-850836-7-iscsi devstack@keystone.service[81435]: *** uWSGI is running in multiple interpreter mode *** Sep 15 12:09:39.390380 dv-cinder-850836-7-iscsi devstack@keystone.service[81435]: spawned uWSGI master process (pid: 81435) Sep 15 12:09:39.390584 dv-cinder-850836-7-iscsi systemd[1]: Started Devstack devstack@keystone.service. Sep 15 12:09:39.391780 dv-cinder-850836-7-iscsi devstack@keystone.service[81435]: spawned uWSGI worker 1 (pid: 81448, cores: 1) Sep 15 12:09:39.391780 dv-cinder-850836-7-iscsi devstack@keystone.service[81435]: spawned uWSGI worker 2 (pid: 81449, cores: 1) Sep 15 12:09:39.391780 dv-cinder-850836-7-iscsi devstack@keystone.service[81435]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Sep 15 12:09:43.632532 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 15 12:09:43.632532 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: from cryptography.utils import int_from_bytes Sep 15 12:09:43.633377 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 15 12:09:43.633377 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: from cryptography.utils import int_from_bytes Sep 15 12:09:43.638749 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 15 12:09:43.638749 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: from cryptography.utils import int_from_bytes Sep 15 12:09:43.639211 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Sep 15 12:09:43.639211 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: from cryptography.utils import int_from_bytes Sep 15 12:09:43.682965 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Sep 15 12:09:43.683154 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Sep 15 12:09:43.683341 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] command line args: [] {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Sep 15 12:09:43.683608 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Sep 15 12:09:43.683780 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] ================================================================================ {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Sep 15 12:09:43.684028 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] admin_token = **** {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 12:09:43.684270 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] config_dir = [] {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 12:09:43.684453 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 12:09:43.684628 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] config_source = [] {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 12:09:43.684828 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] debug = True {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 12:09:43.685026 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 12:09:43.685396 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 12:09:43.685571 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 12:09:43.685820 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 12:09:43.686065 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 12:09:43.686290 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] list_limit = None {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 12:09:43.686399 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Sep 15 12:09:43.686510 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] log_config_append = None {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 12:09:43.686622 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Sep 15 12:09:43.686720 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 12:09:43.686821 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] command line args: [] {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Sep 15 12:09:43.686917 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] log_dir = None {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 12:09:43.687018 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] log_file = None {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 12:09:43.687125 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Sep 15 12:09:43.687234 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 12:09:43.687358 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] ================================================================================ {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Sep 15 12:09:43.687479 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 12:09:43.687594 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] admin_token = **** {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 12:09:43.687713 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 12:09:43.687839 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] config_dir = [] {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 12:09:43.687839 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 12:09:43.688061 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 12:09:43.688185 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] config_source = [] {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 12:09:43.688300 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 12:09:43.688416 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] debug = True {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 12:09:43.688537 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 12:09:43.688650 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 12:09:43.688815 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 12:09:43.688941 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 12:09:43.689067 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 12:09:43.689179 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 12:09:43.689300 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 12:09:43.689421 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 12:09:43.689525 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 12:09:43.689643 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 12:09:43.689766 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 12:09:43.689917 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] list_limit = None {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 12:09:43.690036 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 12:09:43.690156 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] log_config_append = None {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 12:09:43.690271 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 12:09:43.690390 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 12:09:43.690390 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] log_dir = None {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 12:09:43.690622 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 12:09:43.690736 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] log_file = None {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 12:09:43.690849 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 12:09:43.690972 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 12:09:43.690972 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 12:09:43.690972 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 12:09:43.691294 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.15/identity {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 12:09:43.691411 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 12:09:43.691582 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] publish_errors = False {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 12:09:43.691707 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 12:09:43.691707 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 12:09:43.691926 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 12:09:43.692040 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 12:09:43.692177 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 12:09:43.692177 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 12:09:43.692409 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 12:09:43.692532 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 12:09:43.692653 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 12:09:43.692790 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 12:09:43.692907 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 12:09:43.693032 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] standard_threads = False {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 12:09:43.693156 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 12:09:43.693278 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 12:09:43.693393 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 12:09:43.693508 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 12:09:43.693621 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 12:09:43.693735 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 12:09:43.693856 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 12:09:43.693977 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] use_journal = False {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 12:09:43.693977 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] use_json = False {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 12:09:43.694218 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 12:09:43.694345 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] use_stderr = False {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 12:09:43.694345 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] use_syslog = False {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 12:09:43.694609 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.15/identity {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 12:09:43.694733 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 12:09:43.694857 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] publish_errors = False {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 12:09:43.694973 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.695088 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 12:09:43.695202 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.695316 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 12:09:43.695436 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.695605 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 12:09:43.695723 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.695844 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 12:09:43.695959 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.696073 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 12:09:43.696187 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.696309 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] standard_threads = False {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 12:09:43.696431 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.696569 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 12:09:43.696715 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.696849 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 12:09:43.696849 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 12:09:43.697075 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.697181 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] use_journal = False {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 12:09:43.697300 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.697420 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] use_json = False {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 12:09:43.697420 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] use_stderr = False {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 12:09:43.697642 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.697768 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] use_syslog = False {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 12:09:43.697889 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.698005 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Sep 15 12:09:43.698118 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.698231 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.698350 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.698477 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.698593 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.698707 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.698826 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.698941 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.699053 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.699175 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.699290 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.699403 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.699516 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.699628 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.699742 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.699878 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.700033 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.700151 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.700264 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.700378 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.700492 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.700617 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.700739 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.700874 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.700988 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.701102 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.701226 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.701418 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.701536 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.701650 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.701769 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.701885 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.702011 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.702133 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.702248 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.702367 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.702482 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.702601 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.702723 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.702723 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.702938 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.703052 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.703166 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.703279 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.703401 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.703516 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.703630 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.703743 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.703862 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.703978 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.704099 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.704220 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.704341 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.704455 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.704576 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.704696 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.704841 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.704958 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.705073 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.705186 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.705305 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.705426 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.705548 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.705664 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.705784 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.705898 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.706011 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.706125 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.706246 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.706362 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.706476 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.706595 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.706709 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.706828 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.707017 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.707136 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.707256 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.707383 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.707499 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.707623 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.707746 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.707861 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.707975 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.708088 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.708203 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.708316 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.708438 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.708553 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.708666 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.708794 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.708910 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.709035 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.709159 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.709259 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.709354 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.709449 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.709543 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.709637 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.709739 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.709836 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.709942 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.710115 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.710769 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.710895 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.711019 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.711135 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.711239 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.711352 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.711466 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.711580 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.711707 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.711823 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.711934 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.712048 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.712162 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.712283 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.712401 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.712528 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.712645 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.712780 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.712897 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.713001 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.713114 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.713240 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.713367 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.713483 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.713597 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.713721 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.713838 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.713951 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.714072 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.714189 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.714304 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.714420 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.714534 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.714656 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.714773 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.714879 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.714981 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.715095 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.715209 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.715332 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.715448 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.715574 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.715574 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.715790 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.715790 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.716014 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.716130 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.716245 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.716366 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.716481 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.716603 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.716726 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.716908 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.717035 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.717152 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.717267 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.717381 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.717505 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.717505 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.717725 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.717840 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.717954 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.718069 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.718192 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.718309 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.718430 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.718545 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.718669 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.718785 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.718975 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.719094 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.719209 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.719330 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.719453 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.719577 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.719707 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.719832 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.719949 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.720065 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.720215 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.720334 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.720449 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.720573 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.720696 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.720838 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.720956 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.721072 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] auth.external = None {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.721223 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.721366 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.721487 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.721603 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.721725 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.721848 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.721964 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.722091 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] auth.password = None {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.722208 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.722324 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] auth.token = None {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.722439 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.722553 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.722695 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.722822 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.722948 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.723073 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.723191 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.723315 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.723434 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.723549 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.723672 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.723797 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.723916 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.724032 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.724149 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.724271 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] credential.caching = True {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.724388 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.724513 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.724631 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] auth.external = None {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.724747 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.724918 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.725034 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.725149 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.725278 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.725403 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.725520 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.725636 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] auth.password = None {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.725751 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.726442 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] auth.token = None {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.726582 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.726702 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.726819 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.726935 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.727052 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.727168 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.727293 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.727411 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.727536 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.727654 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.727772 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.727888 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.728012 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.728131 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.728248 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.728364 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] credential.caching = True {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.728481 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.728860 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.728992 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] federation.caching = True {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.729113 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.729229 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.729346 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.729462 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.729586 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.729717 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.729836 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.729952 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.730068 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.730185 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.730337 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.730464 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.730581 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.730703 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.730821 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.730937 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.731052 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.731176 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.731294 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.731410 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.731526 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.731641 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.731764 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.731882 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] identity.caching = True {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.731999 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.732115 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.732115 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.732333 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.732457 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.732575 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] federation.caching = True {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.732691 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.732833 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.732967 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.733084 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.733213 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.733333 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.733449 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.733554 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.733702 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.733818 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.734016 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.734137 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.734253 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.734371 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.734487 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.734603 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.734727 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.734850 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.734968 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.735084 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.735201 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.735316 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.735442 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.735560 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] identity.caching = True {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.735676 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.735799 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.735916 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.736033 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.736158 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.736275 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.736392 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.736509 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.736630 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.736769 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.736902 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.737021 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.737138 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.737254 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.737369 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.737487 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.737612 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.737730 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.737847 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.737964 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.738080 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.738196 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.738319 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.738438 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.738555 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.738678 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.738795 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.738912 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.739036 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.739155 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.739272 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.739389 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.739505 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.739622 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.739750 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.739867 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.739984 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.740100 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.740216 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.740364 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.740492 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.740610 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.740727 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.740888 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.741004 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.741131 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.741258 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.741375 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.741515 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.741631 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.741748 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.741864 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.741989 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.742107 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.742224 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.742340 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.742456 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.742581 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.742699 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.742816 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.742932 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.743048 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.743163 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.743287 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.743405 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.743521 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.743643 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.743760 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.743877 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.744001 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.744124 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] ldap.user = None {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.744241 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.744357 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.744473 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.744589 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.744716 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.744865 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.744985 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.745101 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.745217 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.745332 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.745457 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.745574 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.745690 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.745806 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.745923 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.746039 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.746163 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.746281 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.746397 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.746512 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.746633 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.746749 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.746873 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.746992 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.747108 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.747223 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.747339 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.747454 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.747578 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.747695 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.747811 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] ldap.user = None {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.747927 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.748044 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.748159 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.748283 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.748401 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.748516 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.748632 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.748748 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.748920 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.749050 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.749169 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.749286 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.749401 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.749517 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.749634 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.749758 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.749875 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.749991 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.750108 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.750224 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.750340 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.750504 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.750622 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.750738 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.750854 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.750970 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.751085 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.751210 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.751328 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.751444 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.751558 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.751674 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.751789 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.751914 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.752031 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.752147 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.752263 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.752379 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.752494 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.752618 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.752736 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.752867 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.752983 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] resource.caching = True {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.753098 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.753214 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.753342 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.753461 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.753577 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.753692 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.753807 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.753922 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.754046 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.754163 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.754163 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.754382 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.754382 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.754597 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.754597 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.754824 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.754940 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.755055 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.755171 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] role.caching = True {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.755286 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.755411 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] role.driver = sql {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.755528 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.755644 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.755759 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.755874 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.755989 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.756117 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.756236 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.756353 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.756468 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.756587 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.756869 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] resource.caching = True {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.756991 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.757131 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.757248 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.757365 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.757482 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.757611 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.757731 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.757847 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.757962 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.758078 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.758194 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.758318 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.758435 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.758553 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.758669 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.758786 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.758902 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.759027 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.759027 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] role.caching = True {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.759246 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.759362 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] role.driver = sql {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.759477 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.759600 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.759725 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.759844 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.759960 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.760075 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.760192 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.760307 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.760464 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.760591 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.760707 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.760837 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.760955 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.761071 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.761200 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.761318 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.761435 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.761551 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.761674 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.761791 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.761915 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.761915 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.762135 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.762251 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.762366 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.762483 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.762607 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.762725 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.762842 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.762958 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.763073 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.763197 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.763315 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.763431 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.763546 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.763662 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] token.caching = True {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.763662 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.763889 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.764007 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.764123 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.764238 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.764354 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.764470 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.764596 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.764713 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.764875 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.764992 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.765108 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.765224 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.765352 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.765471 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.765587 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.765703 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.765819 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.765941 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.766065 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.766183 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.766305 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.766422 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.766537 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.766658 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.766783 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.766902 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.767019 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.767134 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.767263 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.767389 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.767507 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.767631 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.767750 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] token.caching = True {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.767866 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] database.connection = **** {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.767982 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.768097 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.768213 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.768336 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.768454 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.768570 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.768686 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.768814 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.768933 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.769061 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.769180 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.769296 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.769413 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.769528 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.769644 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.769768 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.769886 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.770002 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.770118 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.770240 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.770356 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.770480 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.770632 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.770751 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.770867 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.770982 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.771099 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.771224 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.771342 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.771467 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.771594 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.771712 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.771712 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.771929 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.772045 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.772170 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] database.connection = **** {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.772288 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81449) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Sep 15 12:09:43.772405 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.772405 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.772405 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.772405 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.772405 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.772405 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.772405 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.772405 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.772405 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.773893 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.773893 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.773893 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.773893 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.773893 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.773893 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.773893 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.773893 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.773893 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Sep 15 12:09:43.774837 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81448) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Sep 15 12:09:43.797597 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81449) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 15 12:09:43.797827 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81449) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 15 12:09:43.798039 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81449) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 15 12:09:43.803337 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81448) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 15 12:09:43.803571 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81448) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 15 12:09:43.803796 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81448) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 15 12:09:43.806502 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81449) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 15 12:09:43.812133 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81448) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 15 12:09:43.834076 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81449) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 15 12:09:43.834300 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81449) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 15 12:09:43.838242 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81448) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 15 12:09:43.838473 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81448) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 15 12:09:43.888338 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81449) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 15 12:09:43.889385 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81448) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 15 12:09:43.937317 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81449) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 15 12:09:43.937546 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81449) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 15 12:09:43.937907 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81448) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 15 12:09:43.938139 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81448) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 15 12:09:44.094192 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81448) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 15 12:09:44.094380 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81449) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 15 12:09:44.094466 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81448) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 15 12:09:44.094576 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81449) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 15 12:09:44.094685 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81448) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 15 12:09:44.094828 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81448) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 15 12:09:44.095043 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81448) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 15 12:09:44.095123 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81449) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 15 12:09:44.095255 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81448) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 15 12:09:44.095346 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81449) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 15 12:09:44.095468 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81449) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 15 12:09:44.095682 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81449) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 15 12:09:44.119408 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81448) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 15 12:09:44.119628 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81448) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 15 12:09:44.119788 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81449) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 15 12:09:44.119890 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81448) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 15 12:09:44.120023 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81449) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 15 12:09:44.120107 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81448) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 15 12:09:44.120245 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81449) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 15 12:09:44.120327 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81448) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 15 12:09:44.120455 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81448) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 15 12:09:44.120537 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81449) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 15 12:09:44.120665 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81449) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 15 12:09:44.120746 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81448) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 15 12:09:44.120901 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81449) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 15 12:09:44.121114 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81449) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 15 12:09:44.131706 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81448) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 15 12:09:44.131952 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81449) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 15 12:09:44.144509 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81448) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 15 12:09:44.144682 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81448) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 15 12:09:44.144860 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81449) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 15 12:09:44.144972 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81448) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 15 12:09:44.145092 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81449) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 15 12:09:44.145175 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81448) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 15 12:09:44.145313 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81449) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 15 12:09:44.145398 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81448) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 15 12:09:44.145530 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81449) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 15 12:09:44.145613 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81448) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 15 12:09:44.145736 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81448) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 15 12:09:44.145815 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81449) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 15 12:09:44.145942 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81449) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 15 12:09:44.146150 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81449) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Sep 15 12:09:44.162252 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81448) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 12:09:44.162419 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81449) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 12:09:44.162503 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81448) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 15 12:09:44.162624 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81449) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 15 12:09:44.162786 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81448) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 12:09:44.162959 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81449) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 12:09:44.163041 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81448) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 15 12:09:44.163157 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81449) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Sep 15 12:09:44.163239 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81448) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 12:09:44.163963 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81449) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 12:09:44.164054 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81448) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 12:09:44.164221 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81449) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 12:09:44.164302 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81448) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 12:09:44.164474 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81449) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 12:09:44.164571 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81448) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 12:09:44.164730 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81449) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 12:09:44.174082 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81448) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 12:09:44.174295 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81449) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 12:09:44.174376 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81448) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 12:09:44.174574 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81449) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 12:09:44.174662 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81448) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 12:09:44.174841 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81449) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 12:09:44.174924 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81448) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 12:09:44.175119 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81449) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 12:09:44.182353 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81448) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 15 12:09:44.182576 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81449) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 15 12:09:44.185271 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81448) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 15 12:09:44.185462 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81449) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 15 12:09:44.185544 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81448) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 12:09:44.185744 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81449) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 12:09:44.185824 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81448) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 12:09:44.186024 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81449) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 12:09:44.186116 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81448) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 12:09:44.186305 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81449) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 12:09:44.186387 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81448) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 12:09:44.186585 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81449) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 12:09:44.186665 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81448) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 12:09:44.186838 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81449) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 12:09:44.186920 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81448) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 12:09:44.187100 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81449) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 12:09:44.187181 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81448) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 12:09:44.187364 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81448) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 12:09:44.187451 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81449) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 12:09:44.187625 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81449) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 12:09:44.187704 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81448) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 12:09:44.187890 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81448) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 12:09:44.187968 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81449) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 12:09:44.188150 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81449) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 12:09:44.210596 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81448) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 12:09:44.210888 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81449) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 12:09:44.212049 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81448) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 15 12:09:44.212335 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81448) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 12:09:44.212416 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81449) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 15 12:09:44.212628 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81449) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 12:09:44.217008 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81448) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 15 12:09:44.217292 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81448) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 12:09:44.217373 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81449) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 15 12:09:44.217574 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81448) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 12:09:44.217667 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81449) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 12:09:44.217876 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81449) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 12:09:44.223854 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81448) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 15 12:09:44.224140 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81448) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 12:09:44.224221 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81449) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 15 12:09:44.224452 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81449) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 12:09:44.230518 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81448) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 15 12:09:44.230816 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81448) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 12:09:44.230896 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81449) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 15 12:09:44.231103 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81448) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 12:09:44.231184 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81449) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 12:09:44.231392 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81448) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 12:09:44.231475 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81449) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 12:09:44.231671 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81448) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 12:09:44.231752 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81449) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 12:09:44.231943 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81449) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 12:09:44.232030 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81448) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 12:09:44.232213 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81448) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 12:09:44.232292 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81449) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 12:09:44.232477 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81449) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 12:09:44.232556 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81448) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 12:09:44.232740 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81449) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 12:09:44.248470 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81448) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 12:09:44.248562 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81449) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 12:09:44.248775 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81448) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 12:09:44.248860 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81449) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 12:09:44.252590 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81449) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 15 12:09:44.252724 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81448) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 15 12:09:44.255729 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81449) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 12:09:44.255823 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81448) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 12:09:44.256021 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81449) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 12:09:44.256103 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81448) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 12:09:44.260144 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81449) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 15 12:09:44.260274 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81448) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 15 12:09:44.263222 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81449) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 15 12:09:44.263485 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81448) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 15 12:09:44.266445 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81449) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 12:09:44.266724 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81448) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 12:09:44.266808 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81449) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 12:09:44.267008 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81449) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 12:09:44.267087 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81448) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 12:09:44.267285 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81449) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 12:09:44.267373 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81448) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 12:09:44.267570 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81448) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 12:09:44.267651 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81449) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 12:09:44.267845 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81449) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 12:09:44.267924 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81448) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 12:09:44.268130 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81448) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 12:09:44.283375 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81449) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 15 12:09:44.283661 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81449) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 12:09:44.283782 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81448) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 15 12:09:44.283923 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81449) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 12:09:44.284070 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81448) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 12:09:44.284183 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81449) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 12:09:44.284334 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81448) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 12:09:44.284595 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81448) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 12:09:44.291486 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81449) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 12:09:44.291948 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81448) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 12:09:44.293039 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81449) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 12:09:44.293511 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81448) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 12:09:44.294548 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81449) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 12:09:44.294818 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81449) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 12:09:44.295020 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81448) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 12:09:44.295298 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81448) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 12:09:44.297729 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81449) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 15 12:09:44.298007 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81449) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 12:09:44.298086 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81448) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 15 12:09:44.298292 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81449) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 12:09:44.298372 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81448) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 12:09:44.298567 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81449) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 12:09:44.298650 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81448) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 12:09:44.298944 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81448) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 12:09:44.299202 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81449) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 12:09:44.299335 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81448) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 12:09:44.310776 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81449) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 15 12:09:44.310948 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81448) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 15 12:09:44.311057 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81449) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 12:09:44.311232 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81448) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 12:09:44.311338 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81449) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 12:09:44.311514 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81448) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 12:09:44.311605 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81449) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 12:09:44.311798 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81448) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 12:09:44.311977 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81449) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 12:09:44.312178 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81448) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 12:09:44.312267 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81449) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 12:09:44.312464 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81448) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 12:09:44.312545 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81449) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 12:09:44.312741 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81448) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 12:09:44.327641 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81449) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 15 12:09:44.328038 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81448) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 15 12:09:44.330515 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81449) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 15 12:09:44.330942 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81448) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 15 12:09:44.333428 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81449) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 12:09:44.333859 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81448) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 12:09:44.334852 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81449) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 12:09:44.335336 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81448) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 12:09:44.336334 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81449) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 15 12:09:44.336614 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81449) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 12:09:44.336812 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81448) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 15 12:09:44.337034 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81449) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 12:09:44.337186 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81448) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 12:09:44.337484 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81448) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 12:09:44.343314 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81449) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 12:09:44.344751 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81449) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 15 12:09:44.346171 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81448) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 12:09:44.347605 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81449) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 12:09:44.347994 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81449) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 12:09:44.348076 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81448) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 15 12:09:44.348274 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81449) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 12:09:44.348546 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81449) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 12:09:44.351454 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81448) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 12:09:44.351850 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81448) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 12:09:44.352138 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81448) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 12:09:44.352416 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81448) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 12:09:44.357065 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81449) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 15 12:09:44.357363 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81449) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 12:09:44.357640 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81449) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 12:09:44.360503 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81448) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 15 12:09:44.360827 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81448) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 12:09:44.361114 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81448) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 12:09:44.364407 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81449) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 15 12:09:44.364689 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81449) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 12:09:44.364987 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81449) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 12:09:44.365260 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81449) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 12:09:44.365583 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81449) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 12:09:44.365938 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81449) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 12:09:44.366270 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81449) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 12:09:44.367098 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81449) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 12:09:44.367365 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81449) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 12:09:44.367629 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81449) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 12:09:44.367904 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81449) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 12:09:44.368177 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81449) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 12:09:44.368488 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81449) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 12:09:44.368578 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81448) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Sep 15 12:09:44.368774 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81449) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 12:09:44.368860 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81448) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 12:09:44.369062 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81448) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 12:09:44.369338 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81448) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 12:09:44.369611 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81448) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 12:09:44.369883 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81448) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 12:09:44.370155 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81448) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 12:09:44.370440 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81448) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 12:09:44.370707 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81448) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 12:09:44.370974 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81448) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 12:09:44.371251 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81448) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 12:09:44.371517 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81448) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 12:09:44.371781 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81448) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 12:09:44.372067 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81448) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Sep 15 12:09:44.403074 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x55d1de5849c0 pid: 81449 (default app) Sep 15 12:09:44.406301 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x55d1de5849c0 pid: 81448 (default app) Sep 15 12:09:44.406387 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ddd269-d437-41c4-b187-de8bd5a76789 None None] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:09:44.406604 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ddd269-d437-41c4-b187-de8bd5a76789 None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:09:44.406873 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ddd269-d437-41c4-b187-de8bd5a76789 None None] PATH_INFO: `/v3` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:09:44.410966 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1/1] 10.222.0.15 () {50 vars in 791 bytes} [Thu Sep 15 12:09:44 2022] GET /identity/v3/ => generated 255 bytes in 7 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:09:51.165372 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566bbfae-25df-4b2a-9338-5435c4783ffd None None] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:09:51.165877 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566bbfae-25df-4b2a-9338-5435c4783ffd None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:09:51.165980 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566bbfae-25df-4b2a-9338-5435c4783ffd None None] PATH_INFO: `/` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:09:51.170171 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1/2] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 12:09:51 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:09:51.175266 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e02d155-d2d0-4d6f-99dc-bee34be1c4aa None None] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:09:51.175702 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e02d155-d2d0-4d6f-99dc-bee34be1c4aa None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:09:51.175839 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e02d155-d2d0-4d6f-99dc-bee34be1c4aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:09:51.247669 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: INFO dbcounter [None req-8e02d155-d2d0-4d6f-99dc-bee34be1c4aa None None] Registered counter for database keystone Sep 15 12:09:51.266939 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG oslo_db.sqlalchemy.engines [None req-8e02d155-d2d0-4d6f-99dc-bee34be1c4aa None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81449) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 15 12:09:51.273038 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writer thread running {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 15 12:09:51.449436 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG passlib.handlers.bcrypt [None req-8e02d155-d2d0-4d6f-99dc-bee34be1c4aa None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81449) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Sep 15 12:09:51.449832 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG passlib.handlers.bcrypt [None req-8e02d155-d2d0-4d6f-99dc-bee34be1c4aa None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81449) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Sep 15 12:09:51.507882 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.auth.core [None req-8e02d155-d2d0-4d6f-99dc-bee34be1c4aa None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81449) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:09:51.509447 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-8e02d155-d2d0-4d6f-99dc-bee34be1c4aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:09:51.639332 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 2/3] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 12:09:51 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 465 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Sep 15 12:09:52.078467 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566bbfae-25df-4b2a-9338-5435c4783ffd None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:09:52.147181 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: INFO dbcounter [None req-566bbfae-25df-4b2a-9338-5435c4783ffd None None] Registered counter for database keystone Sep 15 12:09:52.168992 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG oslo_db.sqlalchemy.engines [None req-566bbfae-25df-4b2a-9338-5435c4783ffd None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81448) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Sep 15 12:09:52.175045 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writer thread running {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Sep 15 12:09:52.293406 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566bbfae-25df-4b2a-9338-5435c4783ffd None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:09:52.297524 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16757fc9-6368-4422-803e-b06f849afbf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:09:52.298157 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16757fc9-6368-4422-803e-b06f849afbf4 admin admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:09:52.298439 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16757fc9-6368-4422-803e-b06f849afbf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:09:52.298712 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16757fc9-6368-4422-803e-b06f849afbf4 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:09:52.307421 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16757fc9-6368-4422-803e-b06f849afbf4 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:09:52.358403 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: WARNING keystone.common.rbac_enforcer.enforcer [None req-16757fc9-6368-4422-803e-b06f849afbf4 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Sep 15 12:09:52.358747 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16757fc9-6368-4422-803e-b06f849afbf4 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:09:52.368554 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 2/4] 10.222.0.15 () {58 vars in 1324 bytes} [Thu Sep 15 12:09:52 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 15 12:09:52.374530 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e02d155-d2d0-4d6f-99dc-bee34be1c4aa None None] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:09:52.390838 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e02d155-d2d0-4d6f-99dc-bee34be1c4aa None None] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:09:52.395134 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead047a7-cb03-43ba-9c3b-c88903bda2b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:09:52.396244 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead047a7-cb03-43ba-9c3b-c88903bda2b9 admin admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:09:52.396523 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead047a7-cb03-43ba-9c3b-c88903bda2b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:09:52.396817 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead047a7-cb03-43ba-9c3b-c88903bda2b9 admin admin] PATH_INFO: `/v3/projects/69bea7b802b04b689af0d31b020f8554` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:09:52.398438 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead047a7-cb03-43ba-9c3b-c88903bda2b9 admin admin] RBAC: Authorizing `identity:get_project(project_id=69bea7b802b04b689af0d31b020f8554)` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:09:52.450559 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: WARNING keystone.common.rbac_enforcer.enforcer [None req-ead047a7-cb03-43ba-9c3b-c88903bda2b9 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Sep 15 12:09:52.450898 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead047a7-cb03-43ba-9c3b-c88903bda2b9 admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:09:52.455313 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 3/5] 10.222.0.15 () {56 vars in 1207 bytes} [Thu Sep 15 12:09:52 2022] GET /identity/v3/projects/69bea7b802b04b689af0d31b020f8554 => generated 344 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:09:54.552463 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f1eb46-8a2b-4972-9225-72990f8e0690 None None] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:09:54.553031 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f1eb46-8a2b-4972-9225-72990f8e0690 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:09:54.553112 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f1eb46-8a2b-4972-9225-72990f8e0690 None None] PATH_INFO: `/` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:09:54.557776 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 3/6] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 12:09:54 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 15 12:09:54.563215 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f606dc-42c5-4b70-b32c-4a93522755dc None None] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:09:54.563636 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f606dc-42c5-4b70-b32c-4a93522755dc None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:09:54.563764 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2f606dc-42c5-4b70-b32c-4a93522755dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:09:54.583922 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.auth.core [None req-e2f606dc-42c5-4b70-b32c-4a93522755dc None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81449) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:09:54.585472 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-e2f606dc-42c5-4b70-b32c-4a93522755dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:09:54.593249 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 4/7] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 12:09:54 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Sep 15 12:09:54.860401 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f1eb46-8a2b-4972-9225-72990f8e0690 None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:09:54.872722 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f1eb46-8a2b-4972-9225-72990f8e0690 None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:09:54.876847 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5210f6-4e37-4974-9deb-41be2ee1a69b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:09:54.877447 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5210f6-4e37-4974-9deb-41be2ee1a69b admin admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:09:54.877723 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5210f6-4e37-4974-9deb-41be2ee1a69b admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:09:54.877992 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5210f6-4e37-4974-9deb-41be2ee1a69b admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:09:54.885523 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5210f6-4e37-4974-9deb-41be2ee1a69b admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:09:54.887259 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5210f6-4e37-4974-9deb-41be2ee1a69b admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:09:54.896071 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 4/8] 10.222.0.15 () {58 vars in 1324 bytes} [Thu Sep 15 12:09:54 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 15 12:09:54.901861 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f606dc-42c5-4b70-b32c-4a93522755dc None None] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:09:54.912488 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2f606dc-42c5-4b70-b32c-4a93522755dc None None] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:09:54.918054 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2fb28c9-32c1-4697-8037-80bb1132d7bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:09:54.918689 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fb28c9-32c1-4697-8037-80bb1132d7bf admin admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:09:54.918988 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fb28c9-32c1-4697-8037-80bb1132d7bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:09:54.919261 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fb28c9-32c1-4697-8037-80bb1132d7bf admin admin] PATH_INFO: `/v3/users/c66eff9fe7124011868989f16846a501` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:09:54.920581 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2fb28c9-32c1-4697-8037-80bb1132d7bf admin admin] RBAC: Authorizing `identity:get_user(user_id=c66eff9fe7124011868989f16846a501)` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:09:54.922438 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2fb28c9-32c1-4697-8037-80bb1132d7bf admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:09:54.923963 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 5/9] 10.222.0.15 () {56 vars in 1198 bytes} [Thu Sep 15 12:09:54 2022] GET /identity/v3/users/c66eff9fe7124011868989f16846a501 => generated 247 bytes in 23 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:09:57.113973 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1dca35c-825d-4525-909d-7847c8879578 None None] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:09:57.114494 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1dca35c-825d-4525-909d-7847c8879578 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:09:57.114494 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1dca35c-825d-4525-909d-7847c8879578 None None] PATH_INFO: `/` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:09:57.115842 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 5/10] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 12:09:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:09:57.121572 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1835e4-8371-4463-acbe-171c98d07df5 None None] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:09:57.121998 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1835e4-8371-4463-acbe-171c98d07df5 None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:09:57.122114 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1835e4-8371-4463-acbe-171c98d07df5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:09:57.140087 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.auth.core [None req-2a1835e4-8371-4463-acbe-171c98d07df5 None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81449) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:09:57.141282 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-2a1835e4-8371-4463-acbe-171c98d07df5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:09:57.155185 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af554d6f-7f52-48da-93f7-197f90c5dd99 None None] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:09:57.155474 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af554d6f-7f52-48da-93f7-197f90c5dd99 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:09:57.155757 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af554d6f-7f52-48da-93f7-197f90c5dd99 None None] PATH_INFO: `/` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:09:57.157131 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 6/11] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 12:09:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 15 12:09:57.162016 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32d456e-2479-430c-96d0-81c1b94e8f62 None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:09:57.162295 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32d456e-2479-430c-96d0-81c1b94e8f62 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:09:57.162565 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32d456e-2479-430c-96d0-81c1b94e8f62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:09:57.203427 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG passlib.handlers.bcrypt [None req-c32d456e-2479-430c-96d0-81c1b94e8f62 None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81448) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Sep 15 12:09:57.203824 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG passlib.handlers.bcrypt [None req-c32d456e-2479-430c-96d0-81c1b94e8f62 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81448) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Sep 15 12:09:57.220802 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 6/12] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 12:09:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 15 12:09:57.253429 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-c32d456e-2479-430c-96d0-81c1b94e8f62 None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:09:57.254934 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-c32d456e-2479-430c-96d0-81c1b94e8f62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:09:57.351940 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 7/13] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 12:09:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 15 12:09:57.489654 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1835e4-8371-4463-acbe-171c98d07df5 None None] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:09:57.515221 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:09:57.596058 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1835e4-8371-4463-acbe-171c98d07df5 None None] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:09:57.628159 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32d456e-2479-430c-96d0-81c1b94e8f62 None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:09:57.629055 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d21ac3-96b0-4b28-8431-a8c394d3ce05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:09:57.629707 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d21ac3-96b0-4b28-8431-a8c394d3ce05 None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:09:57.629996 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d21ac3-96b0-4b28-8431-a8c394d3ce05 None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:09:57.630274 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d21ac3-96b0-4b28-8431-a8c394d3ce05 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:09:57.642330 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d21ac3-96b0-4b28-8431-a8c394d3ce05 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 12:09:57.651693 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application [None req-62d21ac3-96b0-4b28-8431-a8c394d3ce05 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 15 12:09:57.651693 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:09:57.651693 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:09:57.651693 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:09:57.651693 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:09:57.651693 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:09:57.651693 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:09:57.651693 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:09:57.651693 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:09:57.651693 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:09:57.651693 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:09:57.651693 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:09:57.651693 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 12:09:57.651693 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 12:09:57.651693 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 12:09:57.651693 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 12:09:57.652700 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 12:09:57.652700 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 12:09:57.652700 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 12:09:57.652700 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 12:09:57.652700 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:09:57.652700 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:09:57.652700 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:09:57.652700 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:09:57.652700 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:09:57.652700 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:09:57.652700 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:09:57.652700 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application with Lock( Sep 15 12:09:57.652700 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:09:57.652700 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:09:57.652700 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:09:57.652700 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:09:57.653982 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:09:57.653982 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:09:57.653982 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:09:57.653982 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:09:57.653982 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 12:09:57.653982 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 12:09:57.653982 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 12:09:57.653982 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 12:09:57.653982 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 12:09:57.653982 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 12:09:57.653982 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 15 12:09:57.653982 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application Sep 15 12:09:57.653982 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 7/14] 10.222.0.15 () {56 vars in 1096 bytes} [Thu Sep 15 12:09:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:09:57.658473 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d21ac3-96b0-4b28-8431-a8c394d3ce05 None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:09:57.730136 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32d456e-2479-430c-96d0-81c1b94e8f62 None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:09:57.732376 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:09:57.738472 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:09:57.758111 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d21ac3-96b0-4b28-8431-a8c394d3ce05 None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:09:57.766934 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f67589-3924-40a4-a662-ad9e7053a493 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:09:57.767755 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f67589-3924-40a4-a662-ad9e7053a493 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:09:57.768024 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f67589-3924-40a4-a662-ad9e7053a493 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:09:57.768290 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f67589-3924-40a4-a662-ad9e7053a493 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:09:57.791675 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8d0970-e01c-4ebd-aaea-e2c87e27e2f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:09:57.792282 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8d0970-e01c-4ebd-aaea-e2c87e27e2f8 None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:09:57.792563 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8d0970-e01c-4ebd-aaea-e2c87e27e2f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:09:57.792868 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8d0970-e01c-4ebd-aaea-e2c87e27e2f8 None admin] PATH_INFO: `/v3/roles` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:09:57.793527 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8d0970-e01c-4ebd-aaea-e2c87e27e2f8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81449) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:09:57.793832 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8d0970-e01c-4ebd-aaea-e2c87e27e2f8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:09:57.795230 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8d0970-e01c-4ebd-aaea-e2c87e27e2f8 None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:09:57.806563 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 8/15] 10.222.0.15 () {56 vars in 1099 bytes} [Thu Sep 15 12:09:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 12:09:57.812219 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8d0970-e01c-4ebd-aaea-e2c87e27e2f8 None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:09:57.832811 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f67589-3924-40a4-a662-ad9e7053a493 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:09:57.834317 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f67589-3924-40a4-a662-ad9e7053a493 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:09:57.910489 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8d0970-e01c-4ebd-aaea-e2c87e27e2f8 None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:09:57.933756 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 8/16] 10.222.0.15 () {58 vars in 1282 bytes} [Thu Sep 15 12:09:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 15 12:09:57.939603 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f67589-3924-40a4-a662-ad9e7053a493 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:09:57.942265 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:09:57.943473 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2fd030-4512-48dd-b7c3-7cea652c383c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:09:57.944386 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2fd030-4512-48dd-b7c3-7cea652c383c None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:09:57.945500 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2fd030-4512-48dd-b7c3-7cea652c383c None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:09:57.945823 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2fd030-4512-48dd-b7c3-7cea652c383c None admin] PATH_INFO: `/v3/users/c66eff9fe7124011868989f16846a501` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:09:57.946958 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2fd030-4512-48dd-b7c3-7cea652c383c None admin] RBAC: Authorizing `identity:get_user(user_id=c66eff9fe7124011868989f16846a501)` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:09:57.948394 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2fd030-4512-48dd-b7c3-7cea652c383c None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:09:57.949964 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 9/17] 10.222.0.15 () {56 vars in 1177 bytes} [Thu Sep 15 12:09:57 2022] GET /identity/v3/users/c66eff9fe7124011868989f16846a501 => generated 247 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 12:09:57.955957 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2fd030-4512-48dd-b7c3-7cea652c383c None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:09:57.971656 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:09:58.038591 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f67589-3924-40a4-a662-ad9e7053a493 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:09:58.056380 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2fd030-4512-48dd-b7c3-7cea652c383c None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:09:58.071793 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33eebc6e-9bca-4b33-8aeb-f25e3eb09ac8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:09:58.072655 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33eebc6e-9bca-4b33-8aeb-f25e3eb09ac8 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:09:58.072972 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33eebc6e-9bca-4b33-8aeb-f25e3eb09ac8 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:09:58.073246 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33eebc6e-9bca-4b33-8aeb-f25e3eb09ac8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:09:58.088316 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8b4bae-c777-4e94-8c20-da4f8745dc2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:09:58.088963 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8b4bae-c777-4e94-8c20-da4f8745dc2e None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:09:58.089245 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8b4bae-c777-4e94-8c20-da4f8745dc2e None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:09:58.089512 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8b4bae-c777-4e94-8c20-da4f8745dc2e None admin] PATH_INFO: `/v3/domains/default` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:09:58.090607 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8b4bae-c777-4e94-8c20-da4f8745dc2e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:09:58.091377 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33eebc6e-9bca-4b33-8aeb-f25e3eb09ac8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:09:58.092081 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8b4bae-c777-4e94-8c20-da4f8745dc2e None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:09:58.092889 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33eebc6e-9bca-4b33-8aeb-f25e3eb09ac8 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:09:58.093901 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 10/18] 10.222.0.15 () {56 vars in 1108 bytes} [Thu Sep 15 12:09:57 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 12:09:58.099360 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8b4bae-c777-4e94-8c20-da4f8745dc2e None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:09:58.111222 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application [None req-33eebc6e-9bca-4b33-8aeb-f25e3eb09ac8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 12:09:58.111222 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:09:58.111222 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:09:58.111222 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:09:58.111222 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:09:58.111222 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:09:58.111222 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:09:58.111222 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:09:58.111222 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:09:58.111222 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:09:58.111222 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:09:58.111222 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:09:58.111222 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 12:09:58.111222 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 12:09:58.111222 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 12:09:58.111222 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 12:09:58.112107 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:09:58.112107 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:09:58.112107 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:09:58.112107 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:09:58.112107 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:09:58.112107 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:09:58.112107 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:09:58.112107 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application with Lock( Sep 15 12:09:58.112107 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:09:58.112107 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:09:58.112107 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:09:58.112107 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:09:58.112107 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:09:58.112107 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:09:58.112107 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:09:58.112107 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:09:58.112107 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 12:09:58.112971 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 12:09:58.112971 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 12:09:58.112971 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 15 12:09:58.112971 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 9/19] 10.222.0.15 () {56 vars in 1108 bytes} [Thu Sep 15 12:09:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:09:58.118522 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33eebc6e-9bca-4b33-8aeb-f25e3eb09ac8 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:09:58.172982 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:09:58.177403 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:09:58.198008 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8b4bae-c777-4e94-8c20-da4f8745dc2e None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:09:58.219083 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33eebc6e-9bca-4b33-8aeb-f25e3eb09ac8 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:09:58.230932 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639f930f-5015-4c80-bb51-ddde34c22ef3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:09:58.231533 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639f930f-5015-4c80-bb51-ddde34c22ef3 None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:09:58.231814 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639f930f-5015-4c80-bb51-ddde34c22ef3 None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:09:58.232078 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639f930f-5015-4c80-bb51-ddde34c22ef3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:09:58.232813 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639f930f-5015-4c80-bb51-ddde34c22ef3 None admin] RBAC: Adding query filter params (role.id=9c769b5c82884766b2c510b1f0f1364c, scope.domain.id=default, user.id=c66eff9fe7124011868989f16846a501) {{(pid=81449) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:09:58.233119 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639f930f-5015-4c80-bb51-ddde34c22ef3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:09:58.234566 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639f930f-5015-4c80-bb51-ddde34c22ef3 None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:09:58.241208 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 11/20] 10.222.0.15 () {56 vars in 1322 bytes} [Thu Sep 15 12:09:58 2022] GET /identity/v3/role_assignments?user.id=c66eff9fe7124011868989f16846a501&scope.domain.id=default&role.id=9c769b5c82884766b2c510b1f0f1364c => generated 233 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 12:09:58.252357 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b158c071-b3f4-4a9b-bb96-f0558420aaa2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:09:58.253040 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b158c071-b3f4-4a9b-bb96-f0558420aaa2 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:09:58.253323 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b158c071-b3f4-4a9b-bb96-f0558420aaa2 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:09:58.253596 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b158c071-b3f4-4a9b-bb96-f0558420aaa2 None admin] PATH_INFO: `/v3/domains` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:09:58.254263 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b158c071-b3f4-4a9b-bb96-f0558420aaa2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:09:58.254567 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b158c071-b3f4-4a9b-bb96-f0558420aaa2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:09:58.255999 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b158c071-b3f4-4a9b-bb96-f0558420aaa2 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:09:58.276200 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 10/21] 10.222.0.15 () {56 vars in 1109 bytes} [Thu Sep 15 12:09:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 12:10:00.826071 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4339f18-f609-4384-8a8e-b891d371aa43 None None] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:00.826676 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4339f18-f609-4384-8a8e-b891d371aa43 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:00.826676 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4339f18-f609-4384-8a8e-b891d371aa43 None None] PATH_INFO: `/` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:00.828957 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3ef857-daf7-4dab-9231-dcfed3dcd856 None None] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:00.829334 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3ef857-daf7-4dab-9231-dcfed3dcd856 None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:00.829525 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe3ef857-daf7-4dab-9231-dcfed3dcd856 None None] PATH_INFO: `/` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:00.831268 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 12/23] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 12:10:00 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:10:00.831381 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 11/23] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 12:10:00 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:10:00.838029 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53876916-316e-48a7-88e6-8369ff2d6987 None None] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:00.838307 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53876916-316e-48a7-88e6-8369ff2d6987 None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:00.838580 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53876916-316e-48a7-88e6-8369ff2d6987 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:00.842456 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b85a038-f899-4f34-81c3-38526cec74b9 None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:00.842742 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b85a038-f899-4f34-81c3-38526cec74b9 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:00.843016 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b85a038-f899-4f34-81c3-38526cec74b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:00.872878 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-1b85a038-f899-4f34-81c3-38526cec74b9 None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:10:00.874084 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-1b85a038-f899-4f34-81c3-38526cec74b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:10:00.879997 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.auth.core [None req-53876916-316e-48a7-88e6-8369ff2d6987 None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81449) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:10:00.881179 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-53876916-316e-48a7-88e6-8369ff2d6987 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:10:00.995150 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 12/24] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 12:10:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 12:10:00.996427 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae632eef-662a-446f-a5b0-f9fa0d38546e None None] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:00.996715 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae632eef-662a-446f-a5b0-f9fa0d38546e None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:00.997025 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae632eef-662a-446f-a5b0-f9fa0d38546e None None] PATH_INFO: `/` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:00.999774 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 13/25] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 12:10:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:10:01.006249 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6aad75-73eb-4172-935e-6d5a6cd26a69 None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:01.006538 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6aad75-73eb-4172-935e-6d5a6cd26a69 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:01.006818 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6aad75-73eb-4172-935e-6d5a6cd26a69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:01.013006 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 13/26] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 12:10:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 12:10:01.038043 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-be6aad75-73eb-4172-935e-6d5a6cd26a69 None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:10:01.039834 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-be6aad75-73eb-4172-935e-6d5a6cd26a69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:10:01.093149 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:10:01.126069 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 14/27] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 12:10:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 12:10:01.505573 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53876916-316e-48a7-88e6-8369ff2d6987 None None] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:10:01.575337 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6aad75-73eb-4172-935e-6d5a6cd26a69 None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:10:01.625118 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:10:01.681854 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53876916-316e-48a7-88e6-8369ff2d6987 None None] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:10:01.747846 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6aad75-73eb-4172-935e-6d5a6cd26a69 None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:10:01.761661 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22fa837-fff8-42f7-a015-cb3b32d84e30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:10:01.762589 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22fa837-fff8-42f7-a015-cb3b32d84e30 None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:01.763072 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22fa837-fff8-42f7-a015-cb3b32d84e30 None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:01.763538 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22fa837-fff8-42f7-a015-cb3b32d84e30 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:01.781414 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22fa837-fff8-42f7-a015-cb3b32d84e30 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:10:01.785107 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22fa837-fff8-42f7-a015-cb3b32d84e30 None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:10:01.812736 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application [None req-b22fa837-fff8-42f7-a015-cb3b32d84e30 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 12:10:01.812736 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:10:01.812736 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:10:01.812736 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:10:01.812736 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:10:01.812736 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:10:01.812736 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:10:01.812736 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:10:01.812736 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:10:01.812736 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:10:01.812736 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:10:01.812736 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:10:01.812736 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 12:10:01.812736 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 12:10:01.812736 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 12:10:01.812736 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 12:10:01.813657 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:10:01.813657 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:10:01.813657 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:10:01.813657 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:10:01.813657 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:10:01.813657 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:10:01.813657 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:10:01.813657 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application with Lock( Sep 15 12:10:01.813657 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:10:01.813657 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:10:01.813657 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:10:01.813657 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:10:01.813657 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:10:01.813657 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:10:01.813657 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:10:01.813657 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:10:01.813657 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 12:10:01.814458 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 12:10:01.814458 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 12:10:01.814458 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application Sep 15 12:10:01.818818 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 14/28] 10.222.0.15 () {56 vars in 1108 bytes} [Thu Sep 15 12:10:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 314 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:10:01.821236 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22fa837-fff8-42f7-a015-cb3b32d84e30 None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:10:01.821835 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71f874e-a760-4212-955e-a8e17bd8e96e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:10:01.822711 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71f874e-a760-4212-955e-a8e17bd8e96e None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:01.823182 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71f874e-a760-4212-955e-a8e17bd8e96e None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:01.823641 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71f874e-a760-4212-955e-a8e17bd8e96e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:01.840598 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f71f874e-a760-4212-955e-a8e17bd8e96e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 12:10:01.847167 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application [None req-f71f874e-a760-4212-955e-a8e17bd8e96e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 15 12:10:01.847167 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:10:01.847167 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:10:01.847167 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:10:01.847167 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:10:01.847167 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:10:01.847167 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:10:01.847167 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:10:01.847167 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:10:01.847167 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:10:01.847167 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:10:01.847167 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:10:01.847167 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 12:10:01.847167 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 12:10:01.847167 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 12:10:01.847167 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 12:10:01.848019 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 12:10:01.848019 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 12:10:01.848019 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 12:10:01.848019 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 12:10:01.848019 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:10:01.848019 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:10:01.848019 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:10:01.848019 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:10:01.848019 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:10:01.848019 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:10:01.848019 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:10:01.848019 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application with Lock( Sep 15 12:10:01.848019 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:10:01.848019 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:10:01.848019 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:10:01.848019 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:10:01.848793 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:10:01.848793 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:10:01.848793 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:10:01.848793 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:10:01.848793 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 12:10:01.848793 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 12:10:01.848793 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 12:10:01.848793 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 12:10:01.848793 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 12:10:01.848793 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 12:10:01.848793 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 15 12:10:01.848793 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 15 12:10:01.848793 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 15/29] 10.222.0.15 () {56 vars in 1096 bytes} [Thu Sep 15 12:10:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 274 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:10:01.849995 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50eac91-92b2-4bca-8754-1e40d96eaf4e None None] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:01.849995 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50eac91-92b2-4bca-8754-1e40d96eaf4e None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:01.850244 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50eac91-92b2-4bca-8754-1e40d96eaf4e None None] PATH_INFO: `/` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:01.852380 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 16/30] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 12:10:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:10:01.853140 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50eac91-92b2-4bca-8754-1e40d96eaf4e None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:10:01.941183 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:10:01.949762 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22fa837-fff8-42f7-a015-cb3b32d84e30 None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:10:01.963807 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50eac91-92b2-4bca-8754-1e40d96eaf4e None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:10:01.983815 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:10:01.993650 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b86e3ddf-e84f-4d23-a6b8-aad24896fd24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:10:01.994259 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86e3ddf-e84f-4d23-a6b8-aad24896fd24 None admin] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:01.994538 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86e3ddf-e84f-4d23-a6b8-aad24896fd24 None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:01.994829 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b86e3ddf-e84f-4d23-a6b8-aad24896fd24 None admin] PATH_INFO: `/v3/roles` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:01.995587 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86e3ddf-e84f-4d23-a6b8-aad24896fd24 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:10:01.997010 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b86e3ddf-e84f-4d23-a6b8-aad24896fd24 None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:10:02.012702 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de80a249-3cba-4ad6-b0cb-1823ec12eff6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:10:02.013775 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de80a249-3cba-4ad6-b0cb-1823ec12eff6 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:02.014256 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de80a249-3cba-4ad6-b0cb-1823ec12eff6 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:02.014743 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de80a249-3cba-4ad6-b0cb-1823ec12eff6 None admin] PATH_INFO: `/v3/domains` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:02.015582 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de80a249-3cba-4ad6-b0cb-1823ec12eff6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:10:02.016093 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de80a249-3cba-4ad6-b0cb-1823ec12eff6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:10:02.017714 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de80a249-3cba-4ad6-b0cb-1823ec12eff6 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:10:02.035032 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 17/31] 10.222.0.15 () {56 vars in 1109 bytes} [Thu Sep 15 12:10:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:10:02.036221 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de80a249-3cba-4ad6-b0cb-1823ec12eff6 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:10:02.124732 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 15/32] 10.222.0.15 () {60 vars in 1131 bytes} [Thu Sep 15 12:10:01 2022] POST /identity/v3/roles => generated 219 bytes in 305 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 12:10:02.126192 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbb92ea-8826-4984-8dbc-686d7e455fa8 None None] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:02.126485 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbb92ea-8826-4984-8dbc-686d7e455fa8 None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:02.126763 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbb92ea-8826-4984-8dbc-686d7e455fa8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:02.138083 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:10:02.145558 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de80a249-3cba-4ad6-b0cb-1823ec12eff6 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:10:02.165476 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.auth.core [None req-fcbb92ea-8826-4984-8dbc-686d7e455fa8 None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81449) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:10:02.167082 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-fcbb92ea-8826-4984-8dbc-686d7e455fa8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:10:02.192445 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a78c58c-20c0-432a-b8c1-d8d8636022af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:10:02.193445 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a78c58c-20c0-432a-b8c1-d8d8636022af None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:02.193927 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a78c58c-20c0-432a-b8c1-d8d8636022af None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:02.194424 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a78c58c-20c0-432a-b8c1-d8d8636022af None admin] PATH_INFO: `/v3/roles` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:02.195821 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a78c58c-20c0-432a-b8c1-d8d8636022af None admin] RBAC: Adding query filter params (name=admin) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:10:02.196414 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a78c58c-20c0-432a-b8c1-d8d8636022af None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:10:02.198197 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a78c58c-20c0-432a-b8c1-d8d8636022af None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:10:02.213588 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 18/33] 10.222.0.15 () {56 vars in 1099 bytes} [Thu Sep 15 12:10:02 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:10:02.214928 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a78c58c-20c0-432a-b8c1-d8d8636022af None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:10:02.253474 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 16/34] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 12:10:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 12:10:02.262623 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59190e4a-1f13-439d-aa46-7c60aa6676bf None None] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:02.263083 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59190e4a-1f13-439d-aa46-7c60aa6676bf None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:02.263415 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59190e4a-1f13-439d-aa46-7c60aa6676bf None None] PATH_INFO: `/` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:02.265093 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 17/35] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 12:10:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:10:02.266075 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59190e4a-1f13-439d-aa46-7c60aa6676bf None None] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:10:02.337769 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a78c58c-20c0-432a-b8c1-d8d8636022af None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:10:02.379705 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d537afe1-02db-4239-9395-f7db419e5817 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:10:02.380341 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d537afe1-02db-4239-9395-f7db419e5817 None admin] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:02.380620 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d537afe1-02db-4239-9395-f7db419e5817 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:02.380990 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d537afe1-02db-4239-9395-f7db419e5817 None admin] PATH_INFO: `/v3/projects` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:02.382022 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d537afe1-02db-4239-9395-f7db419e5817 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:10:02.384160 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d537afe1-02db-4239-9395-f7db419e5817 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:10:02.431461 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59190e4a-1f13-439d-aa46-7c60aa6676bf None None] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:10:02.452272 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:10:02.480986 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9202b9-6812-4d50-aaf9-f28b871f3a96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:10:02.481652 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9202b9-6812-4d50-aaf9-f28b871f3a96 None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:02.481936 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9202b9-6812-4d50-aaf9-f28b871f3a96 None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:02.482265 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9202b9-6812-4d50-aaf9-f28b871f3a96 None admin] PATH_INFO: `/v3/users/c66eff9fe7124011868989f16846a501` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:02.483475 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9202b9-6812-4d50-aaf9-f28b871f3a96 None admin] RBAC: Authorizing `identity:get_user(user_id=c66eff9fe7124011868989f16846a501)` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:10:02.485227 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9202b9-6812-4d50-aaf9-f28b871f3a96 None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:10:02.488954 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 18/36] 10.222.0.15 () {56 vars in 1177 bytes} [Thu Sep 15 12:10:02 2022] GET /identity/v3/users/c66eff9fe7124011868989f16846a501 => generated 247 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:10:02.490384 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7b647f-bf61-4e58-bfc4-a484e2a67ebe None None] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:02.490680 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7b647f-bf61-4e58-bfc4-a484e2a67ebe None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:02.491054 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7b647f-bf61-4e58-bfc4-a484e2a67ebe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:02.514474 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.auth.core [None req-7b7b647f-bf61-4e58-bfc4-a484e2a67ebe None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81449) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:10:02.515580 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-7b7b647f-bf61-4e58-bfc4-a484e2a67ebe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:10:02.556674 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 19/37] 10.222.0.15 () {60 vars in 1141 bytes} [Thu Sep 15 12:10:02 2022] POST /identity/v3/projects => generated 301 bytes in 342 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 12:10:02.558223 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d537afe1-02db-4239-9395-f7db419e5817 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:10:02.578701 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:10:02.612027 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 19/38] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 12:10:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 12:10:02.612927 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7b647f-bf61-4e58-bfc4-a484e2a67ebe None None] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:10:02.692135 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d537afe1-02db-4239-9395-f7db419e5817 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:10:02.736108 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ef23c7-8c6c-4a89-a451-0bac9eb822a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:10:02.737179 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ef23c7-8c6c-4a89-a451-0bac9eb822a1 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:02.737708 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ef23c7-8c6c-4a89-a451-0bac9eb822a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:02.738203 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ef23c7-8c6c-4a89-a451-0bac9eb822a1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:02.739698 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ef23c7-8c6c-4a89-a451-0bac9eb822a1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:10:02.742089 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ef23c7-8c6c-4a89-a451-0bac9eb822a1 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:10:02.744024 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 20/39] 10.222.0.15 () {56 vars in 1108 bytes} [Thu Sep 15 12:10:02 2022] GET /identity/v3/domains/default => generated 199 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:10:02.746258 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03b8058-a7b2-4bc6-8eb8-88b48152e4e6 None None] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:02.747028 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03b8058-a7b2-4bc6-8eb8-88b48152e4e6 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:02.747526 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03b8058-a7b2-4bc6-8eb8-88b48152e4e6 None None] PATH_INFO: `/` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:02.748997 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 21/40] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 12:10:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:10:02.749880 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03b8058-a7b2-4bc6-8eb8-88b48152e4e6 None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:10:02.757914 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7b647f-bf61-4e58-bfc4-a484e2a67ebe None None] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:10:02.776273 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:10:02.804743 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf6ca20-0ea9-4e38-92d1-6690ed3a720e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:10:02.805446 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf6ca20-0ea9-4e38-92d1-6690ed3a720e None admin] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:02.805730 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf6ca20-0ea9-4e38-92d1-6690ed3a720e None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:02.806004 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf6ca20-0ea9-4e38-92d1-6690ed3a720e None admin] PATH_INFO: `/v3/roles` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:02.806677 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf6ca20-0ea9-4e38-92d1-6690ed3a720e None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:10:02.808070 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf6ca20-0ea9-4e38-92d1-6690ed3a720e None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:10:02.826722 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 20/41] 10.222.0.15 () {60 vars in 1131 bytes} [Thu Sep 15 12:10:02 2022] POST /identity/v3/roles => generated 225 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 12:10:02.828317 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a998f22e-70a2-4713-837d-2cdeb72c9ca4 None None] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:02.828977 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a998f22e-70a2-4713-837d-2cdeb72c9ca4 None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:02.829462 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a998f22e-70a2-4713-837d-2cdeb72c9ca4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:02.848639 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.auth.core [None req-a998f22e-70a2-4713-837d-2cdeb72c9ca4 None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81449) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:10:02.850113 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-a998f22e-70a2-4713-837d-2cdeb72c9ca4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:10:02.877816 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:10:02.894484 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03b8058-a7b2-4bc6-8eb8-88b48152e4e6 None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:10:02.939114 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 21/42] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 12:10:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 12:10:02.940355 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a998f22e-70a2-4713-837d-2cdeb72c9ca4 None None] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:10:02.946000 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f01118-fe70-442a-90ee-6f05135d12ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:10:02.946754 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f01118-fe70-442a-90ee-6f05135d12ad None admin] REQUEST_METHOD: `PUT` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:02.947042 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f01118-fe70-442a-90ee-6f05135d12ad None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:02.947321 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f01118-fe70-442a-90ee-6f05135d12ad None admin] PATH_INFO: `/v3/domains/default/users/c66eff9fe7124011868989f16846a501/roles/9c769b5c82884766b2c510b1f0f1364c` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:02.949808 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f01118-fe70-442a-90ee-6f05135d12ad None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=c66eff9fe7124011868989f16846a501, role_id=9c769b5c82884766b2c510b1f0f1364c)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:10:02.951282 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f01118-fe70-442a-90ee-6f05135d12ad None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:10:02.974882 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 22/43] 10.222.0.15 () {58 vars in 1361 bytes} [Thu Sep 15 12:10:02 2022] PUT /identity/v3/domains/default/users/c66eff9fe7124011868989f16846a501/roles/9c769b5c82884766b2c510b1f0f1364c => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:10:03.177746 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a998f22e-70a2-4713-837d-2cdeb72c9ca4 None None] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:10:03.181030 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:10:03.215570 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32500adf-10cd-4466-9e16-e4f9e0770f33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:10:03.216536 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32500adf-10cd-4466-9e16-e4f9e0770f33 None admin] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:03.217059 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32500adf-10cd-4466-9e16-e4f9e0770f33 None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:03.217554 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32500adf-10cd-4466-9e16-e4f9e0770f33 None admin] PATH_INFO: `/v3/roles` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:03.218478 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32500adf-10cd-4466-9e16-e4f9e0770f33 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:10:03.220070 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32500adf-10cd-4466-9e16-e4f9e0770f33 None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:10:03.236437 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 22/44] 10.222.0.15 () {60 vars in 1131 bytes} [Thu Sep 15 12:10:02 2022] POST /identity/v3/roles => generated 223 bytes in 297 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 12:10:03.251241 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f01118-fe70-442a-90ee-6f05135d12ad None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:10:03.363374 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f01118-fe70-442a-90ee-6f05135d12ad None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:10:03.401042 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98edebd5-dd85-4ad1-b48d-7348305a1bec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:10:03.401709 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98edebd5-dd85-4ad1-b48d-7348305a1bec None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:03.401994 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98edebd5-dd85-4ad1-b48d-7348305a1bec None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:03.402273 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98edebd5-dd85-4ad1-b48d-7348305a1bec None admin] PATH_INFO: `/v3/domains/default` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:03.403410 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98edebd5-dd85-4ad1-b48d-7348305a1bec None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:10:03.404908 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98edebd5-dd85-4ad1-b48d-7348305a1bec None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:10:03.406659 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 23/45] 10.222.0.15 () {56 vars in 1108 bytes} [Thu Sep 15 12:10:03 2022] GET /identity/v3/domains/default => generated 199 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:10:03.412229 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32500adf-10cd-4466-9e16-e4f9e0770f33 None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:10:03.516682 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32500adf-10cd-4466-9e16-e4f9e0770f33 None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:10:03.550063 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80c50ae-0a7e-4cd1-bd5f-29cff8e64923 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:10:03.550677 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80c50ae-0a7e-4cd1-bd5f-29cff8e64923 None admin] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:03.550960 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80c50ae-0a7e-4cd1-bd5f-29cff8e64923 None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:03.551232 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80c50ae-0a7e-4cd1-bd5f-29cff8e64923 None admin] PATH_INFO: `/v3/projects` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:03.551887 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a80c50ae-0a7e-4cd1-bd5f-29cff8e64923 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:10:03.553380 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a80c50ae-0a7e-4cd1-bd5f-29cff8e64923 None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:10:03.581211 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 23/46] 10.222.0.15 () {60 vars in 1141 bytes} [Thu Sep 15 12:10:03 2022] POST /identity/v3/projects => generated 312 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 12:10:05.152790 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fd20b2-cf8f-4286-abf3-84d908c3d9a5 None None] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:05.153306 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fd20b2-cf8f-4286-abf3-84d908c3d9a5 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:05.153414 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fd20b2-cf8f-4286-abf3-84d908c3d9a5 None None] PATH_INFO: `/` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:05.154721 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 24/47] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 12:10:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:10:05.159923 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d0d568-a422-4653-a221-4699ebf13ca0 None None] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:05.160312 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d0d568-a422-4653-a221-4699ebf13ca0 None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:05.160610 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d0d568-a422-4653-a221-4699ebf13ca0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:05.179667 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.auth.core [None req-14d0d568-a422-4653-a221-4699ebf13ca0 None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81449) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:10:05.180886 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-14d0d568-a422-4653-a221-4699ebf13ca0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:10:05.191347 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:10:05.249596 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 24/48] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 12:10:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 12:10:05.519300 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41fd20b2-cf8f-4286-abf3-84d908c3d9a5 None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:10:05.530740 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:10:05.628306 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41fd20b2-cf8f-4286-abf3-84d908c3d9a5 None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:10:05.666876 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef41cf1b-f755-43fe-a644-0215c6445434 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:10:05.667527 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef41cf1b-f755-43fe-a644-0215c6445434 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:05.667813 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef41cf1b-f755-43fe-a644-0215c6445434 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:05.668090 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef41cf1b-f755-43fe-a644-0215c6445434 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:05.679077 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef41cf1b-f755-43fe-a644-0215c6445434 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 12:10:05.684230 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application [None req-ef41cf1b-f755-43fe-a644-0215c6445434 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 15 12:10:05.684230 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:10:05.684230 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:10:05.684230 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:10:05.684230 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:10:05.684230 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:10:05.684230 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:10:05.684230 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:10:05.684230 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:10:05.684230 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:10:05.684230 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:10:05.684230 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:10:05.684230 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 12:10:05.684230 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 12:10:05.684230 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 12:10:05.684230 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 12:10:05.685601 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 12:10:05.685601 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 12:10:05.685601 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 12:10:05.685601 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 12:10:05.685601 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:10:05.685601 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:10:05.685601 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:10:05.685601 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:10:05.685601 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:10:05.685601 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:10:05.685601 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:10:05.685601 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application with Lock( Sep 15 12:10:05.685601 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:10:05.685601 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:10:05.685601 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:10:05.685601 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:10:05.687104 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:10:05.687104 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:10:05.687104 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:10:05.687104 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:10:05.687104 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 12:10:05.687104 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 12:10:05.687104 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 12:10:05.687104 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 12:10:05.687104 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 12:10:05.687104 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 12:10:05.687104 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 15 12:10:05.687104 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 15 12:10:05.687104 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 25/49] 10.222.0.15 () {56 vars in 1096 bytes} [Thu Sep 15 12:10:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:10:05.691142 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d0d568-a422-4653-a221-4699ebf13ca0 None None] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:10:05.746109 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fead677-2bd0-4073-a175-2c600c2227a2 None None] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:05.746403 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fead677-2bd0-4073-a175-2c600c2227a2 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:05.746687 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fead677-2bd0-4073-a175-2c600c2227a2 None None] PATH_INFO: `/` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:05.748650 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 26/50] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 12:10:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 15 12:10:05.753887 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1da0068-974d-4c92-b107-be8ea7b3ec26 None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:05.754177 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1da0068-974d-4c92-b107-be8ea7b3ec26 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:05.754454 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1da0068-974d-4c92-b107-be8ea7b3ec26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:05.774466 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-a1da0068-974d-4c92-b107-be8ea7b3ec26 None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:10:05.776669 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-a1da0068-974d-4c92-b107-be8ea7b3ec26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:10:05.800258 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d0d568-a422-4653-a221-4699ebf13ca0 None None] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:10:05.835631 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a36001-1ace-4571-ad99-f5594ec85571 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:10:05.836251 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a36001-1ace-4571-ad99-f5594ec85571 None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:05.836534 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a36001-1ace-4571-ad99-f5594ec85571 None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:05.836826 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a36001-1ace-4571-ad99-f5594ec85571 None admin] PATH_INFO: `/v3/roles` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:05.837498 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a36001-1ace-4571-ad99-f5594ec85571 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81449) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:10:05.837807 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a36001-1ace-4571-ad99-f5594ec85571 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:10:05.839253 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a36001-1ace-4571-ad99-f5594ec85571 None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:10:05.841477 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:10:05.848041 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 25/51] 10.222.0.15 () {56 vars in 1099 bytes} [Thu Sep 15 12:10:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:10:05.849618 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 27/52] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 12:10:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 15 12:10:05.854152 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a36001-1ace-4571-ad99-f5594ec85571 None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:10:05.865229 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:10:05.960226 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a36001-1ace-4571-ad99-f5594ec85571 None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:10:06.000148 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95fae338-ab33-479b-9db7-655b2c7126a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:10:06.000824 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fae338-ab33-479b-9db7-655b2c7126a9 None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:06.001113 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fae338-ab33-479b-9db7-655b2c7126a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:06.001386 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fae338-ab33-479b-9db7-655b2c7126a9 None admin] PATH_INFO: `/v3/users/c66eff9fe7124011868989f16846a501` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:06.002548 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95fae338-ab33-479b-9db7-655b2c7126a9 None admin] RBAC: Authorizing `identity:get_user(user_id=c66eff9fe7124011868989f16846a501)` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:10:06.004000 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95fae338-ab33-479b-9db7-655b2c7126a9 None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:10:06.005592 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 26/53] 10.222.0.15 () {56 vars in 1177 bytes} [Thu Sep 15 12:10:05 2022] GET /identity/v3/users/c66eff9fe7124011868989f16846a501 => generated 247 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 12:10:06.011042 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1da0068-974d-4c92-b107-be8ea7b3ec26 None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:10:06.115585 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1da0068-974d-4c92-b107-be8ea7b3ec26 None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:10:06.119219 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95fae338-ab33-479b-9db7-655b2c7126a9 None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:10:06.150701 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff357db-43b6-448e-9f9b-b0c2a0fd968c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:10:06.151339 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff357db-43b6-448e-9f9b-b0c2a0fd968c None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:06.151620 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff357db-43b6-448e-9f9b-b0c2a0fd968c None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:06.151890 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff357db-43b6-448e-9f9b-b0c2a0fd968c None admin] PATH_INFO: `/v3/domains/default` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:06.153124 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff357db-43b6-448e-9f9b-b0c2a0fd968c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:10:06.154606 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff357db-43b6-448e-9f9b-b0c2a0fd968c None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:10:06.156157 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 28/54] 10.222.0.15 () {56 vars in 1108 bytes} [Thu Sep 15 12:10:06 2022] GET /identity/v3/domains/default => generated 199 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:10:06.161963 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff357db-43b6-448e-9f9b-b0c2a0fd968c None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:10:06.198298 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:10:06.223637 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:10:06.224186 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95fae338-ab33-479b-9db7-655b2c7126a9 None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:10:06.258392 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577b4d72-12d0-4b98-bdb5-1e921cfa20ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:10:06.259025 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577b4d72-12d0-4b98-bdb5-1e921cfa20ed None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:06.259304 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577b4d72-12d0-4b98-bdb5-1e921cfa20ed None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:06.259576 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577b4d72-12d0-4b98-bdb5-1e921cfa20ed None admin] PATH_INFO: `/v3/domains/default` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:06.260679 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577b4d72-12d0-4b98-bdb5-1e921cfa20ed None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:10:06.261511 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff357db-43b6-448e-9f9b-b0c2a0fd968c None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:10:06.262146 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577b4d72-12d0-4b98-bdb5-1e921cfa20ed None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:10:06.263709 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 27/55] 10.222.0.15 () {56 vars in 1108 bytes} [Thu Sep 15 12:10:06 2022] GET /identity/v3/domains/default => generated 199 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 12:10:06.269398 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577b4d72-12d0-4b98-bdb5-1e921cfa20ed None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:10:06.374628 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577b4d72-12d0-4b98-bdb5-1e921cfa20ed None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:10:06.398652 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1bea918-bb56-43ee-9741-3a11bc2af3f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:10:06.399271 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bea918-bb56-43ee-9741-3a11bc2af3f2 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:06.399558 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bea918-bb56-43ee-9741-3a11bc2af3f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:06.399838 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bea918-bb56-43ee-9741-3a11bc2af3f2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:06.400571 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1bea918-bb56-43ee-9741-3a11bc2af3f2 None admin] RBAC: Adding query filter params (role.id=9c769b5c82884766b2c510b1f0f1364c, scope.domain.id=default, user.id=c66eff9fe7124011868989f16846a501) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:10:06.400924 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1bea918-bb56-43ee-9741-3a11bc2af3f2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:10:06.402366 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1bea918-bb56-43ee-9741-3a11bc2af3f2 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:10:06.409028 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c142a954-8730-4a40-8214-a1d0d45fbce8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:10:06.409641 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c142a954-8730-4a40-8214-a1d0d45fbce8 None admin] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:06.409927 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c142a954-8730-4a40-8214-a1d0d45fbce8 None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:06.410073 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 29/56] 10.222.0.15 () {56 vars in 1322 bytes} [Thu Sep 15 12:10:06 2022] GET /identity/v3/role_assignments?user.id=c66eff9fe7124011868989f16846a501&scope.domain.id=default&role.id=9c769b5c82884766b2c510b1f0f1364c => generated 530 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 12:10:06.410499 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c142a954-8730-4a40-8214-a1d0d45fbce8 None admin] PATH_INFO: `/v3/projects` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:06.411186 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c142a954-8730-4a40-8214-a1d0d45fbce8 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:10:06.412652 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c142a954-8730-4a40-8214-a1d0d45fbce8 None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:10:06.429797 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:10:06.442626 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 28/57] 10.222.0.15 () {60 vars in 1140 bytes} [Thu Sep 15 12:10:06 2022] POST /identity/v3/projects => generated 298 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 15 12:10:08.604007 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abcccb9-aefc-43d1-bdfd-d7881a429989 None None] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:08.604834 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abcccb9-aefc-43d1-bdfd-d7881a429989 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:08.604917 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abcccb9-aefc-43d1-bdfd-d7881a429989 None None] PATH_INFO: `/` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:08.606268 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 30/58] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 12:10:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:10:08.611672 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8efe95b-16f3-417e-b9b9-cb83d6a53815 None None] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:08.612070 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8efe95b-16f3-417e-b9b9-cb83d6a53815 None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:08.612225 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8efe95b-16f3-417e-b9b9-cb83d6a53815 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:08.630642 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.auth.core [None req-f8efe95b-16f3-417e-b9b9-cb83d6a53815 None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81449) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:10:08.631831 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-f8efe95b-16f3-417e-b9b9-cb83d6a53815 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:10:08.695897 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 29/59] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 12:10:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 15 12:10:08.961574 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abcccb9-aefc-43d1-bdfd-d7881a429989 None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:10:09.058171 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abcccb9-aefc-43d1-bdfd-d7881a429989 None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:10:09.083139 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:10:09.091401 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d545a0-88e3-4e38-9338-118849c63b14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:10:09.092036 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d545a0-88e3-4e38-9338-118849c63b14 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:09.092319 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d545a0-88e3-4e38-9338-118849c63b14 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:09.092593 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d545a0-88e3-4e38-9338-118849c63b14 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:09.093730 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d545a0-88e3-4e38-9338-118849c63b14 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:10:09.095149 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d545a0-88e3-4e38-9338-118849c63b14 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:10:09.096685 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 31/60] 10.222.0.15 () {56 vars in 1108 bytes} [Thu Sep 15 12:10:08 2022] GET /identity/v3/domains/default => generated 199 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:10:09.102047 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8efe95b-16f3-417e-b9b9-cb83d6a53815 None None] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:10:09.197068 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8efe95b-16f3-417e-b9b9-cb83d6a53815 None None] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:10:09.220951 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:10:09.229865 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f7b199-4a0c-4e00-804b-f0908bfd13bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:10:09.230463 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f7b199-4a0c-4e00-804b-f0908bfd13bb None admin] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:09.230744 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f7b199-4a0c-4e00-804b-f0908bfd13bb None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:09.231015 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f7b199-4a0c-4e00-804b-f0908bfd13bb None admin] PATH_INFO: `/v3/users` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:09.231652 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f7b199-4a0c-4e00-804b-f0908bfd13bb None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:10:09.233107 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f7b199-4a0c-4e00-804b-f0908bfd13bb None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:10:09.259981 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 30/61] 10.222.0.15 () {60 vars in 1132 bytes} [Thu Sep 15 12:10:09 2022] POST /identity/v3/users => generated 275 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 12:10:12.372903 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a2821d-fc65-4e2c-83b7-5a646ef1af6e None None] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:12.373873 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a2821d-fc65-4e2c-83b7-5a646ef1af6e None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:12.374378 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a2821d-fc65-4e2c-83b7-5a646ef1af6e None None] PATH_INFO: `/` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:12.375929 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 32/62] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 12:10:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:10:12.381782 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e6249d-985e-4b59-8b32-0615cbf78c85 None None] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:12.382892 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e6249d-985e-4b59-8b32-0615cbf78c85 None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:12.382892 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e6249d-985e-4b59-8b32-0615cbf78c85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:12.406522 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.auth.core [None req-55e6249d-985e-4b59-8b32-0615cbf78c85 None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81449) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:10:12.407740 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-55e6249d-985e-4b59-8b32-0615cbf78c85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:10:12.488495 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 31/63] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 12:10:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 15 12:10:12.554901 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df98a3b-a375-47a9-b8f0-33d9d31c1ae2 None None] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:12.555511 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df98a3b-a375-47a9-b8f0-33d9d31c1ae2 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:12.556000 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df98a3b-a375-47a9-b8f0-33d9d31c1ae2 None None] PATH_INFO: `/` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:12.557673 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 33/64] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 12:10:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:10:12.563526 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e98858-a164-474d-899e-5d4a30d27fcc None None] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:12.564069 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e98858-a164-474d-899e-5d4a30d27fcc None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:12.564543 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e98858-a164-474d-899e-5d4a30d27fcc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:12.593185 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.auth.core [None req-56e98858-a164-474d-899e-5d4a30d27fcc None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81449) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:10:12.594749 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-56e98858-a164-474d-899e-5d4a30d27fcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:10:12.692484 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 32/65] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 12:10:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 12:10:12.797779 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df98a3b-a375-47a9-b8f0-33d9d31c1ae2 None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:10:12.940806 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df98a3b-a375-47a9-b8f0-33d9d31c1ae2 None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:10:12.958805 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b454fa-3616-4b6f-bf5e-31c2ab69b17c None None] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:12.959407 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b454fa-3616-4b6f-bf5e-31c2ab69b17c None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:12.959885 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b454fa-3616-4b6f-bf5e-31c2ab69b17c None None] PATH_INFO: `/` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:12.962014 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 33/66] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 12:10:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 15 12:10:12.969534 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e16212-48c1-4923-8b53-1363b2d8efb7 None None] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:12.970079 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e16212-48c1-4923-8b53-1363b2d8efb7 None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:12.970574 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e16212-48c1-4923-8b53-1363b2d8efb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:12.997666 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.auth.core [None req-70e16212-48c1-4923-8b53-1363b2d8efb7 None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81449) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:10:12.998855 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-70e16212-48c1-4923-8b53-1363b2d8efb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:10:13.020410 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0ec7b2-f110-4153-837a-bfaab528af5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:10:13.021187 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0ec7b2-f110-4153-837a-bfaab528af5f None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:13.021474 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0ec7b2-f110-4153-837a-bfaab528af5f None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:13.021751 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0ec7b2-f110-4153-837a-bfaab528af5f None admin] PATH_INFO: `/v3/roles/member` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:13.033534 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0ec7b2-f110-4153-837a-bfaab528af5f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 12:10:13.038972 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application [None req-4b0ec7b2-f110-4153-837a-bfaab528af5f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 15 12:10:13.038972 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:10:13.038972 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:10:13.038972 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:10:13.038972 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:10:13.038972 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:10:13.038972 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:10:13.038972 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:10:13.038972 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:10:13.038972 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:10:13.038972 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:10:13.038972 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:10:13.038972 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 12:10:13.038972 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 12:10:13.038972 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 12:10:13.038972 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 12:10:13.039854 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 12:10:13.039854 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 12:10:13.039854 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 12:10:13.039854 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 12:10:13.039854 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:10:13.039854 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:10:13.039854 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:10:13.039854 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:10:13.039854 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:10:13.039854 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:10:13.039854 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:10:13.039854 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application with Lock( Sep 15 12:10:13.039854 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:10:13.039854 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:10:13.039854 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:10:13.039854 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:10:13.040615 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:10:13.040615 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:10:13.040615 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:10:13.040615 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:10:13.040615 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 12:10:13.040615 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 12:10:13.040615 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 12:10:13.040615 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 12:10:13.040615 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 12:10:13.040615 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 12:10:13.040615 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 15 12:10:13.040615 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 15 12:10:13.043234 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 34/67] 10.222.0.15 () {56 vars in 1099 bytes} [Thu Sep 15 12:10:12 2022] GET /identity/v3/roles/member => generated 84 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:10:13.049135 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0ec7b2-f110-4153-837a-bfaab528af5f None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:10:13.049638 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:10:13.090258 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 34/68] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 12:10:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 15 12:10:13.091619 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e16212-48c1-4923-8b53-1363b2d8efb7 None None] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:10:13.117579 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:10:13.166990 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0ec7b2-f110-4153-837a-bfaab528af5f None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:10:13.208288 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e16212-48c1-4923-8b53-1363b2d8efb7 None None] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:10:13.217885 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39dc2b0-e246-46f4-a3d9-89346f05c327 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:10:13.218945 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39dc2b0-e246-46f4-a3d9-89346f05c327 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:13.219439 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39dc2b0-e246-46f4-a3d9-89346f05c327 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:13.219924 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39dc2b0-e246-46f4-a3d9-89346f05c327 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:13.243870 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39dc2b0-e246-46f4-a3d9-89346f05c327 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 12:10:13.248820 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application [None req-f39dc2b0-e246-46f4-a3d9-89346f05c327 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 15 12:10:13.248820 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:10:13.248820 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:10:13.248820 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:10:13.248820 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:10:13.248820 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:10:13.248820 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:10:13.248820 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:10:13.248820 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:10:13.248820 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:10:13.248820 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:10:13.248820 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:10:13.248820 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 12:10:13.248820 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 12:10:13.248820 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 12:10:13.248820 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 12:10:13.249676 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 12:10:13.249676 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 12:10:13.249676 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 12:10:13.249676 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 12:10:13.249676 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:10:13.249676 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:10:13.249676 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:10:13.249676 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:10:13.249676 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:10:13.249676 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:10:13.249676 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:10:13.249676 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application with Lock( Sep 15 12:10:13.249676 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:10:13.249676 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:10:13.249676 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:10:13.249676 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:10:13.250441 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:10:13.250441 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:10:13.250441 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:10:13.250441 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:10:13.250441 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 12:10:13.250441 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 12:10:13.250441 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 12:10:13.250441 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 12:10:13.250441 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 12:10:13.250441 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 12:10:13.250441 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 15 12:10:13.250441 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 15 12:10:13.250441 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 35/69] 10.222.0.15 () {56 vars in 1096 bytes} [Thu Sep 15 12:10:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:10:13.256742 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39dc2b0-e246-46f4-a3d9-89346f05c327 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:10:13.262439 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93106bb-3ff3-4590-9521-15c93dd4deca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:10:13.263325 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93106bb-3ff3-4590-9521-15c93dd4deca None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:13.263819 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93106bb-3ff3-4590-9521-15c93dd4deca None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:13.264289 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93106bb-3ff3-4590-9521-15c93dd4deca None admin] PATH_INFO: `/v3/roles` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:13.265360 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93106bb-3ff3-4590-9521-15c93dd4deca None admin] RBAC: Adding query filter params (name=member) {{(pid=81449) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:10:13.265986 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93106bb-3ff3-4590-9521-15c93dd4deca None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:10:13.267567 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93106bb-3ff3-4590-9521-15c93dd4deca None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:10:13.277585 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 35/70] 10.222.0.15 () {56 vars in 1101 bytes} [Thu Sep 15 12:10:13 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:10:13.283532 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93106bb-3ff3-4590-9521-15c93dd4deca None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:10:13.325402 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:10:13.410194 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93106bb-3ff3-4590-9521-15c93dd4deca None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:10:13.428403 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39dc2b0-e246-46f4-a3d9-89346f05c327 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:10:13.445859 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931b8a31-9b4a-4601-9c14-51f8ecfd09f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:10:13.446500 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931b8a31-9b4a-4601-9c14-51f8ecfd09f9 None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:13.446781 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931b8a31-9b4a-4601-9c14-51f8ecfd09f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:13.447054 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931b8a31-9b4a-4601-9c14-51f8ecfd09f9 None admin] PATH_INFO: `/v3/users/6147f6de110b41cd99def2cbeb937ad1` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:13.447995 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:10:13.460303 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-931b8a31-9b4a-4601-9c14-51f8ecfd09f9 None admin] RBAC: Authorizing `identity:get_user(user_id=6147f6de110b41cd99def2cbeb937ad1)` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:10:13.461792 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-931b8a31-9b4a-4601-9c14-51f8ecfd09f9 None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:10:13.463387 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 36/71] 10.222.0.15 () {56 vars in 1177 bytes} [Thu Sep 15 12:10:13 2022] GET /identity/v3/users/6147f6de110b41cd99def2cbeb937ad1 => generated 275 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:10:13.464595 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748ed805-e323-4a68-ac14-f0e5ce3d5fa4 None None] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:13.464950 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748ed805-e323-4a68-ac14-f0e5ce3d5fa4 None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:13.465178 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748ed805-e323-4a68-ac14-f0e5ce3d5fa4 None None] PATH_INFO: `/` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:13.466010 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ec05b7-2b89-4add-b3fc-dc6e872a3e3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:10:13.466525 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 37/72] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 12:10:13 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:10:13.467129 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748ed805-e323-4a68-ac14-f0e5ce3d5fa4 None None] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:10:13.469116 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ec05b7-2b89-4add-b3fc-dc6e872a3e3f None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:13.469366 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ec05b7-2b89-4add-b3fc-dc6e872a3e3f None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:13.469639 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ec05b7-2b89-4add-b3fc-dc6e872a3e3f None admin] PATH_INFO: `/v3/roles` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:13.470602 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ec05b7-2b89-4add-b3fc-dc6e872a3e3f None admin] RBAC: Adding query filter params (name=admin) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:10:13.471710 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ec05b7-2b89-4add-b3fc-dc6e872a3e3f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:10:13.473121 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ec05b7-2b89-4add-b3fc-dc6e872a3e3f None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:10:13.482489 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 36/73] 10.222.0.15 () {56 vars in 1099 bytes} [Thu Sep 15 12:10:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:10:13.483681 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72002e6-411b-47a6-ab9c-b611c866ba8a None None] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:13.484017 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72002e6-411b-47a6-ab9c-b611c866ba8a None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:13.484277 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72002e6-411b-47a6-ab9c-b611c866ba8a None None] PATH_INFO: `/` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:13.485510 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 37/74] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 12:10:13 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:10:13.486160 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72002e6-411b-47a6-ab9c-b611c866ba8a None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:10:13.560813 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:10:13.570738 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748ed805-e323-4a68-ac14-f0e5ce3d5fa4 None None] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:10:13.587208 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72002e6-411b-47a6-ab9c-b611c866ba8a None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:10:13.604282 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c96d649-3535-4a84-aed5-80fdabe3c4a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:10:13.605198 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c96d649-3535-4a84-aed5-80fdabe3c4a2 None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:13.605485 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c96d649-3535-4a84-aed5-80fdabe3c4a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:13.605762 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c96d649-3535-4a84-aed5-80fdabe3c4a2 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:13.617921 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c96d649-3535-4a84-aed5-80fdabe3c4a2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 12:10:13.620868 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6846a466-8968-453e-9db9-7cc6bf4d7f1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:10:13.621481 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6846a466-8968-453e-9db9-7cc6bf4d7f1c None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:13.621789 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6846a466-8968-453e-9db9-7cc6bf4d7f1c None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:13.622063 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6846a466-8968-453e-9db9-7cc6bf4d7f1c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:13.622704 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application [None req-1c96d649-3535-4a84-aed5-80fdabe3c4a2 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 15 12:10:13.622704 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:10:13.622704 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:10:13.622704 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:10:13.622704 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:10:13.622704 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:10:13.622704 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:10:13.622704 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:10:13.622704 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:10:13.622704 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:10:13.622704 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:10:13.622704 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:10:13.622704 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 12:10:13.622704 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 12:10:13.622704 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 12:10:13.622704 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 12:10:13.623712 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 12:10:13.623712 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 12:10:13.623712 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 12:10:13.623712 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 12:10:13.623712 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:10:13.623712 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:10:13.623712 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:10:13.623712 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:10:13.623712 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:10:13.623712 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:10:13.623712 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:10:13.623712 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application with Lock( Sep 15 12:10:13.623712 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:10:13.623712 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:10:13.623712 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:10:13.623712 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:10:13.624499 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:10:13.624499 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:10:13.624499 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:10:13.624499 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:10:13.624499 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 12:10:13.624499 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 12:10:13.624499 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 12:10:13.624499 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 12:10:13.624499 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 12:10:13.624499 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 12:10:13.624499 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 15 12:10:13.624499 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application Sep 15 12:10:13.624499 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 38/75] 10.222.0.15 () {56 vars in 1114 bytes} [Thu Sep 15 12:10:13 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:10:13.625539 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7df4908-e846-4bca-8ac3-fd48d71513cd None None] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:13.625828 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7df4908-e846-4bca-8ac3-fd48d71513cd None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:13.626126 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7df4908-e846-4bca-8ac3-fd48d71513cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:13.642604 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.auth.core [None req-a7df4908-e846-4bca-8ac3-fd48d71513cd None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81449) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:10:13.643833 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-a7df4908-e846-4bca-8ac3-fd48d71513cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:10:13.673563 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:10:13.695573 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6846a466-8968-453e-9db9-7cc6bf4d7f1c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:10:13.697063 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6846a466-8968-453e-9db9-7cc6bf4d7f1c None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:10:13.709874 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 39/76] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 12:10:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 12:10:13.711255 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7df4908-e846-4bca-8ac3-fd48d71513cd None None] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:10:13.796606 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:10:13.802908 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 38/77] 10.222.0.15 () {58 vars in 1282 bytes} [Thu Sep 15 12:10:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 15 12:10:13.802908 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da371b2-2c0e-4bf9-932a-2d6b94077201 None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:13.802908 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da371b2-2c0e-4bf9-932a-2d6b94077201 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:13.802908 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da371b2-2c0e-4bf9-932a-2d6b94077201 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:13.818101 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-3da371b2-2c0e-4bf9-932a-2d6b94077201 None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:10:13.819289 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-3da371b2-2c0e-4bf9-932a-2d6b94077201 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:10:13.822258 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7df4908-e846-4bca-8ac3-fd48d71513cd None None] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:10:13.856195 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dce31e7-f916-4c2f-a89c-9cfe3f2b0418 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:10:13.856848 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dce31e7-f916-4c2f-a89c-9cfe3f2b0418 None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:13.857132 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dce31e7-f916-4c2f-a89c-9cfe3f2b0418 None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:13.857408 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dce31e7-f916-4c2f-a89c-9cfe3f2b0418 None admin] PATH_INFO: `/v3/users/c66eff9fe7124011868989f16846a501` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:13.858515 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dce31e7-f916-4c2f-a89c-9cfe3f2b0418 None admin] RBAC: Authorizing `identity:get_user(user_id=c66eff9fe7124011868989f16846a501)` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:10:13.860068 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dce31e7-f916-4c2f-a89c-9cfe3f2b0418 None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:10:13.866232 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 40/78] 10.222.0.15 () {56 vars in 1177 bytes} [Thu Sep 15 12:10:13 2022] GET /identity/v3/users/c66eff9fe7124011868989f16846a501 => generated 247 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:10:13.866232 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dce31e7-f916-4c2f-a89c-9cfe3f2b0418 None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:10:13.897022 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 39/79] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 12:10:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 12:10:13.898573 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da371b2-2c0e-4bf9-932a-2d6b94077201 None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:10:13.910211 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:10:13.994821 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dce31e7-f916-4c2f-a89c-9cfe3f2b0418 None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:10:14.023580 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da371b2-2c0e-4bf9-932a-2d6b94077201 None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:10:14.035855 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce61992-99b7-4167-9d5e-b9ee2f685a09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:10:14.036473 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce61992-99b7-4167-9d5e-b9ee2f685a09 None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:14.036789 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce61992-99b7-4167-9d5e-b9ee2f685a09 None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:14.038439 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce61992-99b7-4167-9d5e-b9ee2f685a09 None admin] PATH_INFO: `/v3/roles` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:14.039108 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce61992-99b7-4167-9d5e-b9ee2f685a09 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81449) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:10:14.039418 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce61992-99b7-4167-9d5e-b9ee2f685a09 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:10:14.040824 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce61992-99b7-4167-9d5e-b9ee2f685a09 None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:10:14.048963 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 41/80] 10.222.0.15 () {56 vars in 1111 bytes} [Thu Sep 15 12:10:13 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:10:14.049505 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce61992-99b7-4167-9d5e-b9ee2f685a09 None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:10:14.065753 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:10:14.067393 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f7039e-d61d-458b-81af-cdf9cd2b7325 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:10:14.068167 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f7039e-d61d-458b-81af-cdf9cd2b7325 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:14.068461 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f7039e-d61d-458b-81af-cdf9cd2b7325 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:14.068814 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f7039e-d61d-458b-81af-cdf9cd2b7325 None admin] PATH_INFO: `/v3/projects/10a21ffb0b8d4800a805c76f171c40c1` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:14.070701 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f7039e-d61d-458b-81af-cdf9cd2b7325 None admin] RBAC: Authorizing `identity:get_project(project_id=10a21ffb0b8d4800a805c76f171c40c1)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:10:14.072287 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f7039e-d61d-458b-81af-cdf9cd2b7325 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:10:14.074254 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 40/81] 10.222.0.15 () {56 vars in 1186 bytes} [Thu Sep 15 12:10:13 2022] GET /identity/v3/projects/10a21ffb0b8d4800a805c76f171c40c1 => generated 298 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:10:14.074744 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f7039e-d61d-458b-81af-cdf9cd2b7325 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:10:14.155850 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:10:14.159555 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce61992-99b7-4167-9d5e-b9ee2f685a09 None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:10:14.180253 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f7039e-d61d-458b-81af-cdf9cd2b7325 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:10:14.193129 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0774fa-04bd-41d0-814b-333e6360fc2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:10:14.193755 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0774fa-04bd-41d0-814b-333e6360fc2b None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:14.194046 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0774fa-04bd-41d0-814b-333e6360fc2b None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:14.194337 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0774fa-04bd-41d0-814b-333e6360fc2b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:14.213934 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637cd5e2-7ff6-439b-82aa-81ea6b8a7008 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:10:14.214588 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637cd5e2-7ff6-439b-82aa-81ea6b8a7008 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:14.214863 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637cd5e2-7ff6-439b-82aa-81ea6b8a7008 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:14.215143 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637cd5e2-7ff6-439b-82aa-81ea6b8a7008 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:14.216270 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637cd5e2-7ff6-439b-82aa-81ea6b8a7008 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:10:14.217759 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637cd5e2-7ff6-439b-82aa-81ea6b8a7008 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:10:14.219325 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 41/82] 10.222.0.15 () {56 vars in 1108 bytes} [Thu Sep 15 12:10:14 2022] GET /identity/v3/domains/default => generated 199 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:10:14.220090 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637cd5e2-7ff6-439b-82aa-81ea6b8a7008 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:10:14.260598 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e0774fa-04bd-41d0-814b-333e6360fc2b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:10:14.262093 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e0774fa-04bd-41d0-814b-333e6360fc2b None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:10:14.277363 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:10:14.321627 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637cd5e2-7ff6-439b-82aa-81ea6b8a7008 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:10:14.355706 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88386039-b851-41c6-9704-6848447eff8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:10:14.356349 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88386039-b851-41c6-9704-6848447eff8b None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:14.356633 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88386039-b851-41c6-9704-6848447eff8b None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:14.356951 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88386039-b851-41c6-9704-6848447eff8b None admin] PATH_INFO: `/v3/users/6147f6de110b41cd99def2cbeb937ad1` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:14.358099 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88386039-b851-41c6-9704-6848447eff8b None admin] RBAC: Authorizing `identity:get_user(user_id=6147f6de110b41cd99def2cbeb937ad1)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:10:14.359531 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88386039-b851-41c6-9704-6848447eff8b None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:10:14.361140 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 42/83] 10.222.0.15 () {56 vars in 1177 bytes} [Thu Sep 15 12:10:14 2022] GET /identity/v3/users/6147f6de110b41cd99def2cbeb937ad1 => generated 275 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:10:14.361935 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88386039-b851-41c6-9704-6848447eff8b None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:10:14.367855 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 42/84] 10.222.0.15 () {58 vars in 1282 bytes} [Thu Sep 15 12:10:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 15 12:10:14.368681 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0774fa-04bd-41d0-814b-333e6360fc2b None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:10:14.379453 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:10:14.462673 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88386039-b851-41c6-9704-6848447eff8b None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:10:14.465572 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0774fa-04bd-41d0-814b-333e6360fc2b None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:10:14.477957 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:10:14.496003 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39505651-ec40-4966-ba43-a1f3790ff597 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:10:14.496609 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39505651-ec40-4966-ba43-a1f3790ff597 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:14.496919 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39505651-ec40-4966-ba43-a1f3790ff597 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:14.497189 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39505651-ec40-4966-ba43-a1f3790ff597 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:14.497911 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39505651-ec40-4966-ba43-a1f3790ff597 None admin] RBAC: Adding query filter params (role.id=0679d6628a25438d90dc4bdc0401eda2, scope.project.id=10a21ffb0b8d4800a805c76f171c40c1, user.id=6147f6de110b41cd99def2cbeb937ad1) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:10:14.498217 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39505651-ec40-4966-ba43-a1f3790ff597 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:10:14.498892 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89d28ae-4c0f-445a-8c32-b85fec5cb0b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:10:14.499519 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89d28ae-4c0f-445a-8c32-b85fec5cb0b1 None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:14.499669 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39505651-ec40-4966-ba43-a1f3790ff597 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:10:14.499816 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89d28ae-4c0f-445a-8c32-b85fec5cb0b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:14.500065 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89d28ae-4c0f-445a-8c32-b85fec5cb0b1 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:14.505219 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 43/85] 10.222.0.15 () {56 vars in 1374 bytes} [Thu Sep 15 12:10:14 2022] GET /identity/v3/role_assignments?user.id=6147f6de110b41cd99def2cbeb937ad1&scope.project.id=10a21ffb0b8d4800a805c76f171c40c1&role.id=0679d6628a25438d90dc4bdc0401eda2 => generated 259 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:10:14.513231 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39505651-ec40-4966-ba43-a1f3790ff597 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:10:14.517326 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f89d28ae-4c0f-445a-8c32-b85fec5cb0b1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 12:10:14.522406 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application [None req-f89d28ae-4c0f-445a-8c32-b85fec5cb0b1 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 15 12:10:14.522406 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:10:14.522406 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:10:14.522406 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:10:14.522406 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:10:14.522406 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:10:14.522406 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:10:14.522406 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:10:14.522406 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:10:14.522406 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:10:14.522406 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:10:14.522406 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:10:14.522406 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 12:10:14.522406 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 12:10:14.522406 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 12:10:14.522406 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 12:10:14.523359 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 12:10:14.523359 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 12:10:14.523359 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 12:10:14.523359 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 12:10:14.523359 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:10:14.523359 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:10:14.523359 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:10:14.523359 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:10:14.523359 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:10:14.523359 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:10:14.523359 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:10:14.523359 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application with Lock( Sep 15 12:10:14.523359 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:10:14.523359 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:10:14.523359 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:10:14.523359 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:10:14.524206 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:10:14.524206 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:10:14.524206 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:10:14.524206 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:10:14.524206 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 12:10:14.524206 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 12:10:14.524206 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 12:10:14.524206 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 12:10:14.524206 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 12:10:14.524206 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 12:10:14.524206 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 15 12:10:14.524206 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application Sep 15 12:10:14.524206 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 43/86] 10.222.0.15 () {56 vars in 1099 bytes} [Thu Sep 15 12:10:14 2022] GET /identity/v3/roles/member => generated 84 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:10:14.525572 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89d28ae-4c0f-445a-8c32-b85fec5cb0b1 None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:10:14.605458 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:10:14.632977 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39505651-ec40-4966-ba43-a1f3790ff597 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:10:14.640540 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89d28ae-4c0f-445a-8c32-b85fec5cb0b1 None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:10:14.669976 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ce235f-2303-465d-b7bd-c8b43c4f7f64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:10:14.670601 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ce235f-2303-465d-b7bd-c8b43c4f7f64 None admin] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:14.670885 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ce235f-2303-465d-b7bd-c8b43c4f7f64 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:14.671153 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ce235f-2303-465d-b7bd-c8b43c4f7f64 None admin] PATH_INFO: `/v3/users` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:14.671810 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ce235f-2303-465d-b7bd-c8b43c4f7f64 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:10:14.673288 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ce235f-2303-465d-b7bd-c8b43c4f7f64 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:10:14.676176 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd5f4a6-4d22-431c-a071-7fea6d27e5a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:10:14.677031 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd5f4a6-4d22-431c-a071-7fea6d27e5a6 None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:14.677138 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd5f4a6-4d22-431c-a071-7fea6d27e5a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:14.677356 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd5f4a6-4d22-431c-a071-7fea6d27e5a6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:14.713154 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 44/87] 10.222.0.15 () {60 vars in 1132 bytes} [Thu Sep 15 12:10:14 2022] POST /identity/v3/users => generated 289 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 12:10:14.714579 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ce235f-2303-465d-b7bd-c8b43c4f7f64 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:10:14.734557 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:10:14.776435 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd5f4a6-4d22-431c-a071-7fea6d27e5a6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:10:14.778977 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd5f4a6-4d22-431c-a071-7fea6d27e5a6 None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:10:14.864956 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ce235f-2303-465d-b7bd-c8b43c4f7f64 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:10:14.906016 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:10:14.915092 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe274cdd-fc5e-4ec5-8fd5-6dfe47ed5f11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:10:14.916247 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe274cdd-fc5e-4ec5-8fd5-6dfe47ed5f11 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:14.916737 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe274cdd-fc5e-4ec5-8fd5-6dfe47ed5f11 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:14.917293 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe274cdd-fc5e-4ec5-8fd5-6dfe47ed5f11 None admin] PATH_INFO: `/v3/projects/10a21ffb0b8d4800a805c76f171c40c1` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:14.919416 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe274cdd-fc5e-4ec5-8fd5-6dfe47ed5f11 None admin] RBAC: Authorizing `identity:get_project(project_id=10a21ffb0b8d4800a805c76f171c40c1)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:10:14.921322 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe274cdd-fc5e-4ec5-8fd5-6dfe47ed5f11 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:10:14.925728 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 44/88] 10.222.0.15 () {58 vars in 1282 bytes} [Thu Sep 15 12:10:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 15 12:10:14.926064 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd5f4a6-4d22-431c-a071-7fea6d27e5a6 None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:10:14.926311 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 45/89] 10.222.0.15 () {56 vars in 1186 bytes} [Thu Sep 15 12:10:14 2022] GET /identity/v3/projects/10a21ffb0b8d4800a805c76f171c40c1 => generated 298 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:10:14.932047 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe274cdd-fc5e-4ec5-8fd5-6dfe47ed5f11 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:10:15.024417 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:10:15.064060 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd5f4a6-4d22-431c-a071-7fea6d27e5a6 None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:10:15.067078 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe274cdd-fc5e-4ec5-8fd5-6dfe47ed5f11 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:10:15.112498 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8a0229-8cee-4634-aa3c-08ec0fa464cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:10:15.113180 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8a0229-8cee-4634-aa3c-08ec0fa464cd None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:15.113712 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8a0229-8cee-4634-aa3c-08ec0fa464cd None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:15.113996 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8a0229-8cee-4634-aa3c-08ec0fa464cd None admin] PATH_INFO: `/v3/roles` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:15.115394 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8a0229-8cee-4634-aa3c-08ec0fa464cd None admin] RBAC: Adding query filter params (name=member) {{(pid=81449) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:10:15.115662 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8a0229-8cee-4634-aa3c-08ec0fa464cd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:10:15.117863 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8a0229-8cee-4634-aa3c-08ec0fa464cd None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:10:15.127909 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 45/90] 10.222.0.15 () {56 vars in 1101 bytes} [Thu Sep 15 12:10:14 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:10:15.128732 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8a0229-8cee-4634-aa3c-08ec0fa464cd None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:10:15.128871 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8e91e9-2da9-4038-a2ac-a73fc835ebd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:10:15.129433 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8e91e9-2da9-4038-a2ac-a73fc835ebd6 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:15.129720 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8e91e9-2da9-4038-a2ac-a73fc835ebd6 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:15.129996 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8e91e9-2da9-4038-a2ac-a73fc835ebd6 None admin] PATH_INFO: `/v3/projects/10a21ffb0b8d4800a805c76f171c40c1` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:15.131135 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b8e91e9-2da9-4038-a2ac-a73fc835ebd6 None admin] RBAC: Authorizing `identity:get_project(project_id=10a21ffb0b8d4800a805c76f171c40c1)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:10:15.132621 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b8e91e9-2da9-4038-a2ac-a73fc835ebd6 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:10:15.134275 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 46/91] 10.222.0.15 () {56 vars in 1186 bytes} [Thu Sep 15 12:10:14 2022] GET /identity/v3/projects/10a21ffb0b8d4800a805c76f171c40c1 => generated 298 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:10:15.137687 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8e91e9-2da9-4038-a2ac-a73fc835ebd6 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:10:15.225102 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:10:15.245076 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8a0229-8cee-4634-aa3c-08ec0fa464cd None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:10:15.274538 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8e91e9-2da9-4038-a2ac-a73fc835ebd6 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:10:15.282661 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737d6916-5561-4c70-b977-506b5bb14aec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:10:15.283286 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737d6916-5561-4c70-b977-506b5bb14aec None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:15.283574 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737d6916-5561-4c70-b977-506b5bb14aec None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:15.283994 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737d6916-5561-4c70-b977-506b5bb14aec None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:15.284830 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737d6916-5561-4c70-b977-506b5bb14aec None admin] RBAC: Adding query filter params (role.id=9c769b5c82884766b2c510b1f0f1364c, scope.project.id=10a21ffb0b8d4800a805c76f171c40c1, user.id=c66eff9fe7124011868989f16846a501) {{(pid=81449) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:10:15.285153 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737d6916-5561-4c70-b977-506b5bb14aec None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:10:15.286755 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737d6916-5561-4c70-b977-506b5bb14aec None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:10:15.293751 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 46/92] 10.222.0.15 () {56 vars in 1374 bytes} [Thu Sep 15 12:10:15 2022] GET /identity/v3/role_assignments?user.id=c66eff9fe7124011868989f16846a501&scope.project.id=10a21ffb0b8d4800a805c76f171c40c1&role.id=9c769b5c82884766b2c510b1f0f1364c => generated 259 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:10:15.294571 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737d6916-5561-4c70-b977-506b5bb14aec None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:10:15.297539 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:10:15.339583 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9eeaf2-cf3b-400c-ad29-d9d8d60cbaa2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:10:15.340711 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9eeaf2-cf3b-400c-ad29-d9d8d60cbaa2 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:15.341216 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9eeaf2-cf3b-400c-ad29-d9d8d60cbaa2 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:15.341792 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9eeaf2-cf3b-400c-ad29-d9d8d60cbaa2 None admin] PATH_INFO: `/v3/users/6147f6de110b41cd99def2cbeb937ad1` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:15.344010 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f9eeaf2-cf3b-400c-ad29-d9d8d60cbaa2 None admin] RBAC: Authorizing `identity:get_user(user_id=6147f6de110b41cd99def2cbeb937ad1)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:10:15.346029 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f9eeaf2-cf3b-400c-ad29-d9d8d60cbaa2 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:10:15.349431 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 47/93] 10.222.0.15 () {56 vars in 1177 bytes} [Thu Sep 15 12:10:15 2022] GET /identity/v3/users/6147f6de110b41cd99def2cbeb937ad1 => generated 275 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:10:15.357397 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9eeaf2-cf3b-400c-ad29-d9d8d60cbaa2 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:10:15.514372 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737d6916-5561-4c70-b977-506b5bb14aec None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:10:15.556111 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9eeaf2-cf3b-400c-ad29-d9d8d60cbaa2 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:10:15.565495 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c69d214-e8ce-4dd3-9d01-1061c45482c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:10:15.566217 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c69d214-e8ce-4dd3-9d01-1061c45482c8 None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:15.566650 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c69d214-e8ce-4dd3-9d01-1061c45482c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:15.567366 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c69d214-e8ce-4dd3-9d01-1061c45482c8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:15.567721 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c69d214-e8ce-4dd3-9d01-1061c45482c8 None admin] RBAC: Adding query filter params (role.id=1081b93ddfd547649ba8582f7a6e719e, scope.project.id=10a21ffb0b8d4800a805c76f171c40c1, user.id=6147f6de110b41cd99def2cbeb937ad1) {{(pid=81449) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:10:15.568160 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c69d214-e8ce-4dd3-9d01-1061c45482c8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:10:15.570446 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c69d214-e8ce-4dd3-9d01-1061c45482c8 None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:10:15.576748 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 47/94] 10.222.0.15 () {56 vars in 1374 bytes} [Thu Sep 15 12:10:15 2022] GET /identity/v3/role_assignments?user.id=6147f6de110b41cd99def2cbeb937ad1&scope.project.id=10a21ffb0b8d4800a805c76f171c40c1&role.id=1081b93ddfd547649ba8582f7a6e719e => generated 259 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:10:15.609617 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ace386-238e-43ed-9c5b-bf4b69c826b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:10:15.610269 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ace386-238e-43ed-9c5b-bf4b69c826b6 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:15.610554 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ace386-238e-43ed-9c5b-bf4b69c826b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:15.610825 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ace386-238e-43ed-9c5b-bf4b69c826b6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:15.615093 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:10:15.728042 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ace386-238e-43ed-9c5b-bf4b69c826b6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:10:15.729847 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ace386-238e-43ed-9c5b-bf4b69c826b6 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:10:15.873513 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 48/95] 10.222.0.15 () {58 vars in 1282 bytes} [Thu Sep 15 12:10:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 516 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 15 12:10:15.879037 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c69d214-e8ce-4dd3-9d01-1061c45482c8 None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:10:15.980472 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:10:16.033730 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c69d214-e8ce-4dd3-9d01-1061c45482c8 None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:10:16.078983 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8dcc902-984f-4dfa-9fbe-35bd01f86835 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:10:16.079638 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dcc902-984f-4dfa-9fbe-35bd01f86835 None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:16.079917 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dcc902-984f-4dfa-9fbe-35bd01f86835 None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:16.080191 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dcc902-984f-4dfa-9fbe-35bd01f86835 None admin] PATH_INFO: `/v3/projects/53dad4f5e490445fa10339bdf7c51660` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:16.081418 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8dcc902-984f-4dfa-9fbe-35bd01f86835 None admin] RBAC: Authorizing `identity:get_project(project_id=53dad4f5e490445fa10339bdf7c51660)` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:10:16.082878 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8dcc902-984f-4dfa-9fbe-35bd01f86835 None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:10:16.084671 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 48/96] 10.222.0.15 () {56 vars in 1186 bytes} [Thu Sep 15 12:10:15 2022] GET /identity/v3/projects/53dad4f5e490445fa10339bdf7c51660 => generated 312 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:10:16.090445 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ace386-238e-43ed-9c5b-bf4b69c826b6 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:10:16.145921 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:10:16.233495 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ace386-238e-43ed-9c5b-bf4b69c826b6 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:10:16.286858 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b17da9c-86f6-487f-b23c-ba31b4f63f3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:10:16.287488 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b17da9c-86f6-487f-b23c-ba31b4f63f3f None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:16.287769 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b17da9c-86f6-487f-b23c-ba31b4f63f3f None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:16.288041 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b17da9c-86f6-487f-b23c-ba31b4f63f3f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:16.288829 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b17da9c-86f6-487f-b23c-ba31b4f63f3f None admin] RBAC: Adding query filter params (role.id=0679d6628a25438d90dc4bdc0401eda2, scope.project.id=53dad4f5e490445fa10339bdf7c51660, user.id=6147f6de110b41cd99def2cbeb937ad1) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:10:16.289144 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b17da9c-86f6-487f-b23c-ba31b4f63f3f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:10:16.290596 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b17da9c-86f6-487f-b23c-ba31b4f63f3f None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:10:16.301140 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 49/97] 10.222.0.15 () {56 vars in 1374 bytes} [Thu Sep 15 12:10:16 2022] GET /identity/v3/role_assignments?user.id=6147f6de110b41cd99def2cbeb937ad1&scope.project.id=53dad4f5e490445fa10339bdf7c51660&role.id=0679d6628a25438d90dc4bdc0401eda2 => generated 259 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:10:17.005176 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6a5552-fa6a-4712-83f1-ae45d9c1a5af None None] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:17.005676 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6a5552-fa6a-4712-83f1-ae45d9c1a5af None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:17.005745 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6a5552-fa6a-4712-83f1-ae45d9c1a5af None None] PATH_INFO: `/` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:17.007819 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 49/98] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 12:10:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:10:17.013350 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61ce065-acd6-440f-b112-948dbed252f6 None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:17.013845 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61ce065-acd6-440f-b112-948dbed252f6 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:17.014065 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61ce065-acd6-440f-b112-948dbed252f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:17.042791 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-f61ce065-acd6-440f-b112-948dbed252f6 None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:10:17.044000 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-f61ce065-acd6-440f-b112-948dbed252f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:10:17.134254 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 50/99] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 12:10:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 12:10:17.667140 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6a5552-fa6a-4712-83f1-ae45d9c1a5af None None] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:10:17.810393 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6a5552-fa6a-4712-83f1-ae45d9c1a5af None None] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:10:17.846914 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:10:17.857447 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35ebef4-5697-400c-a9fc-4e83b0593306 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:10:17.858107 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35ebef4-5697-400c-a9fc-4e83b0593306 None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:17.858393 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35ebef4-5697-400c-a9fc-4e83b0593306 None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:17.858674 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35ebef4-5697-400c-a9fc-4e83b0593306 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:17.872918 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35ebef4-5697-400c-a9fc-4e83b0593306 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 12:10:17.877661 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application [None req-e35ebef4-5697-400c-a9fc-4e83b0593306 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 15 12:10:17.877661 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:10:17.877661 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:10:17.877661 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:10:17.877661 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:10:17.877661 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:10:17.877661 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:10:17.877661 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:10:17.877661 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:10:17.877661 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:10:17.877661 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:10:17.877661 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:10:17.877661 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 12:10:17.877661 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 12:10:17.877661 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 12:10:17.877661 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 12:10:17.878497 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 12:10:17.878497 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 12:10:17.878497 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 12:10:17.878497 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 12:10:17.878497 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:10:17.878497 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:10:17.878497 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:10:17.878497 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:10:17.878497 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:10:17.878497 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:10:17.878497 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:10:17.878497 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application with Lock( Sep 15 12:10:17.878497 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:10:17.878497 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:10:17.878497 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:10:17.878497 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:10:17.879273 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:10:17.879273 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:10:17.879273 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:10:17.879273 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:10:17.879273 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 12:10:17.879273 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 12:10:17.879273 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 12:10:17.879273 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 12:10:17.879273 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 12:10:17.879273 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 12:10:17.879273 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 15 12:10:17.879273 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application Sep 15 12:10:17.879273 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 50/100] 10.222.0.15 () {56 vars in 1099 bytes} [Thu Sep 15 12:10:17 2022] GET /identity/v3/roles/member => generated 84 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:10:17.884784 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61ce065-acd6-440f-b112-948dbed252f6 None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:10:17.937879 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:10:18.018028 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61ce065-acd6-440f-b112-948dbed252f6 None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:10:18.056861 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5535643f-6485-4b3e-91bb-3c4ce587f935 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:10:18.057494 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5535643f-6485-4b3e-91bb-3c4ce587f935 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:18.057779 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5535643f-6485-4b3e-91bb-3c4ce587f935 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:18.058058 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5535643f-6485-4b3e-91bb-3c4ce587f935 None admin] PATH_INFO: `/v3/roles` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:18.058732 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5535643f-6485-4b3e-91bb-3c4ce587f935 None admin] RBAC: Adding query filter params (name=member) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:10:18.059038 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5535643f-6485-4b3e-91bb-3c4ce587f935 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:10:18.060446 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5535643f-6485-4b3e-91bb-3c4ce587f935 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:10:18.069162 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 51/101] 10.222.0.15 () {56 vars in 1101 bytes} [Thu Sep 15 12:10:17 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:10:18.075218 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35ebef4-5697-400c-a9fc-4e83b0593306 None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:10:18.078134 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f36f8e-48a9-4b2c-9f5d-d8a0232e75d8 None None] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:18.078430 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f36f8e-48a9-4b2c-9f5d-d8a0232e75d8 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:18.078711 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f36f8e-48a9-4b2c-9f5d-d8a0232e75d8 None None] PATH_INFO: `/` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:18.080023 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 52/102] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 12:10:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:10:18.085910 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a19ce4-97af-48a6-a454-11860c1ae75f None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:18.086203 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a19ce4-97af-48a6-a454-11860c1ae75f None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:18.086477 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a19ce4-97af-48a6-a454-11860c1ae75f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:18.129811 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-60a19ce4-97af-48a6-a454-11860c1ae75f None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:10:18.131031 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-60a19ce4-97af-48a6-a454-11860c1ae75f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:10:18.287604 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 53/103] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 12:10:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 15 12:10:18.307622 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35ebef4-5697-400c-a9fc-4e83b0593306 None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:10:18.345921 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8593912-6fd2-4c40-8a3d-822bc91ebd8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:10:18.346796 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8593912-6fd2-4c40-8a3d-822bc91ebd8c None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:18.347257 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8593912-6fd2-4c40-8a3d-822bc91ebd8c None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:18.347673 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8593912-6fd2-4c40-8a3d-822bc91ebd8c None admin] PATH_INFO: `/v3/users/6147f6de110b41cd99def2cbeb937ad1` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:18.348989 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8593912-6fd2-4c40-8a3d-822bc91ebd8c None admin] RBAC: Authorizing `identity:get_user(user_id=6147f6de110b41cd99def2cbeb937ad1)` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:10:18.350572 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8593912-6fd2-4c40-8a3d-822bc91ebd8c None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:10:18.352925 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 51/104] 10.222.0.15 () {56 vars in 1177 bytes} [Thu Sep 15 12:10:18 2022] GET /identity/v3/users/6147f6de110b41cd99def2cbeb937ad1 => generated 275 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:10:18.361868 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a19ce4-97af-48a6-a454-11860c1ae75f None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:10:18.391644 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:10:18.443580 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98abc9f5-973a-475d-b057-fcf8fcdbf17d None None] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:18.443892 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98abc9f5-973a-475d-b057-fcf8fcdbf17d None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:18.444141 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98abc9f5-973a-475d-b057-fcf8fcdbf17d None None] PATH_INFO: `/` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:18.445629 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 52/105] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 12:10:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:10:18.456004 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4674a00-2b67-486f-b562-0bc7226cf06f None None] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:18.456290 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4674a00-2b67-486f-b562-0bc7226cf06f None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:18.456568 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4674a00-2b67-486f-b562-0bc7226cf06f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:18.482493 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.auth.core [None req-a4674a00-2b67-486f-b562-0bc7226cf06f None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81449) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:10:18.484504 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-a4674a00-2b67-486f-b562-0bc7226cf06f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:10:18.500987 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a19ce4-97af-48a6-a454-11860c1ae75f None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:10:18.565508 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3956b18a-edb4-4f0b-a4f8-2b4b0c36c89f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:10:18.566079 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3956b18a-edb4-4f0b-a4f8-2b4b0c36c89f None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:18.566413 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3956b18a-edb4-4f0b-a4f8-2b4b0c36c89f None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:18.566692 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3956b18a-edb4-4f0b-a4f8-2b4b0c36c89f None admin] PATH_INFO: `/v3/projects/10a21ffb0b8d4800a805c76f171c40c1` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:18.570076 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3956b18a-edb4-4f0b-a4f8-2b4b0c36c89f None admin] RBAC: Authorizing `identity:get_project(project_id=10a21ffb0b8d4800a805c76f171c40c1)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:10:18.571517 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3956b18a-edb4-4f0b-a4f8-2b4b0c36c89f None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:10:18.573179 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 54/106] 10.222.0.15 () {56 vars in 1186 bytes} [Thu Sep 15 12:10:18 2022] GET /identity/v3/projects/10a21ffb0b8d4800a805c76f171c40c1 => generated 298 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:10:18.579111 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3956b18a-edb4-4f0b-a4f8-2b4b0c36c89f None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:10:18.598423 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:10:18.619453 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 53/107] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 12:10:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 15 12:10:18.678438 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4674a00-2b67-486f-b562-0bc7226cf06f None None] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:10:18.748331 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:10:18.775705 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3956b18a-edb4-4f0b-a4f8-2b4b0c36c89f None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:10:18.857579 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057f516f-1e8a-4960-b8d1-989e9560ca7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:10:18.858195 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057f516f-1e8a-4960-b8d1-989e9560ca7d None admin] REQUEST_METHOD: `PUT` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:18.858482 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057f516f-1e8a-4960-b8d1-989e9560ca7d None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:18.858761 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057f516f-1e8a-4960-b8d1-989e9560ca7d None admin] PATH_INFO: `/v3/projects/10a21ffb0b8d4800a805c76f171c40c1/users/6147f6de110b41cd99def2cbeb937ad1/roles/0679d6628a25438d90dc4bdc0401eda2` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:18.861546 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057f516f-1e8a-4960-b8d1-989e9560ca7d None admin] RBAC: Authorizing `identity:create_grant(project_id=10a21ffb0b8d4800a805c76f171c40c1, user_id=6147f6de110b41cd99def2cbeb937ad1, role_id=0679d6628a25438d90dc4bdc0401eda2)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:10:18.861963 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4674a00-2b67-486f-b562-0bc7226cf06f None None] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:10:18.862992 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057f516f-1e8a-4960-b8d1-989e9560ca7d None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:10:18.890478 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 55/108] 10.222.0.15 () {58 vars in 1439 bytes} [Thu Sep 15 12:10:18 2022] PUT /identity/v3/projects/10a21ffb0b8d4800a805c76f171c40c1/users/6147f6de110b41cd99def2cbeb937ad1/roles/0679d6628a25438d90dc4bdc0401eda2 => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:10:18.891964 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e44022-dff3-4e0a-b76b-2db5616bc2aa None None] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:18.892260 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e44022-dff3-4e0a-b76b-2db5616bc2aa None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:18.892540 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e44022-dff3-4e0a-b76b-2db5616bc2aa None None] PATH_INFO: `/` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:18.894053 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 56/109] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 12:10:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:10:18.895216 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4506590-3ef2-41eb-a522-99d2a69cc439 None None] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:18.895622 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4506590-3ef2-41eb-a522-99d2a69cc439 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:18.895904 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4506590-3ef2-41eb-a522-99d2a69cc439 None None] PATH_INFO: `/` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:18.898425 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 57/110] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 12:10:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:10:18.902322 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8196dc26-e58b-4c29-b252-162e18ab1581 None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:18.904542 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8196dc26-e58b-4c29-b252-162e18ab1581 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:18.904854 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8196dc26-e58b-4c29-b252-162e18ab1581 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:18.929491 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc85a57-5a10-40a2-8ee2-a9b0b8b0eeba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:10:18.931361 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc85a57-5a10-40a2-8ee2-a9b0b8b0eeba None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:18.931673 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc85a57-5a10-40a2-8ee2-a9b0b8b0eeba None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:18.931955 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc85a57-5a10-40a2-8ee2-a9b0b8b0eeba None admin] PATH_INFO: `/v3/domains/default` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:18.932790 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-8196dc26-e58b-4c29-b252-162e18ab1581 None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:10:18.935017 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc85a57-5a10-40a2-8ee2-a9b0b8b0eeba None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:10:18.935338 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-8196dc26-e58b-4c29-b252-162e18ab1581 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:10:18.938009 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc85a57-5a10-40a2-8ee2-a9b0b8b0eeba None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:10:18.942037 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 54/111] 10.222.0.15 () {56 vars in 1108 bytes} [Thu Sep 15 12:10:18 2022] GET /identity/v3/domains/default => generated 199 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:10:18.945206 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edacf79-8a02-4458-835d-44782c36c8fd None None] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:18.945511 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edacf79-8a02-4458-835d-44782c36c8fd None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:18.945787 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edacf79-8a02-4458-835d-44782c36c8fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:18.983806 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.auth.core [None req-7edacf79-8a02-4458-835d-44782c36c8fd None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81449) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:10:18.986651 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-7edacf79-8a02-4458-835d-44782c36c8fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:10:19.055335 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 58/112] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 12:10:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 12:10:19.056107 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8196dc26-e58b-4c29-b252-162e18ab1581 None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:10:19.099908 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:10:19.119892 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 55/113] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 12:10:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 12:10:19.120686 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7edacf79-8a02-4458-835d-44782c36c8fd None None] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:10:19.143888 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:10:19.167712 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8196dc26-e58b-4c29-b252-162e18ab1581 None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:10:19.203208 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100f9158-b4f6-41fa-9e14-c00681720068 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:10:19.203944 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100f9158-b4f6-41fa-9e14-c00681720068 None admin] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:19.204257 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100f9158-b4f6-41fa-9e14-c00681720068 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:19.204603 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100f9158-b4f6-41fa-9e14-c00681720068 None admin] PATH_INFO: `/v3/projects` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:19.205485 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-100f9158-b4f6-41fa-9e14-c00681720068 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:10:19.206915 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-100f9158-b4f6-41fa-9e14-c00681720068 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:10:19.246880 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7edacf79-8a02-4458-835d-44782c36c8fd None None] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:10:19.252624 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 59/114] 10.222.0.15 () {60 vars in 1141 bytes} [Thu Sep 15 12:10:19 2022] POST /identity/v3/projects => generated 302 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 12:10:19.311855 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4807f58-2869-47fc-9598-4702b033fca1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:10:19.312964 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4807f58-2869-47fc-9598-4702b033fca1 None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:19.313551 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4807f58-2869-47fc-9598-4702b033fca1 None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:19.314094 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4807f58-2869-47fc-9598-4702b033fca1 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:19.327290 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4807f58-2869-47fc-9598-4702b033fca1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 12:10:19.332649 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application [None req-d4807f58-2869-47fc-9598-4702b033fca1 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 15 12:10:19.332649 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:10:19.332649 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:10:19.332649 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:10:19.332649 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:10:19.332649 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:10:19.332649 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:10:19.332649 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:10:19.332649 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:10:19.332649 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:10:19.332649 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:10:19.332649 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:10:19.332649 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 12:10:19.332649 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 12:10:19.332649 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 12:10:19.332649 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 12:10:19.333528 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 12:10:19.333528 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 12:10:19.333528 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 12:10:19.333528 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 12:10:19.333528 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:10:19.333528 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:10:19.333528 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:10:19.333528 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:10:19.333528 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:10:19.333528 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:10:19.333528 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:10:19.333528 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application with Lock( Sep 15 12:10:19.333528 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:10:19.333528 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:10:19.333528 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:10:19.333528 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:10:19.334289 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:10:19.334289 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:10:19.334289 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:10:19.334289 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:10:19.334289 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 12:10:19.334289 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 12:10:19.334289 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 12:10:19.334289 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 12:10:19.334289 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 12:10:19.334289 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 12:10:19.334289 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 15 12:10:19.334289 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application Sep 15 12:10:19.337181 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 56/115] 10.222.0.15 () {56 vars in 1099 bytes} [Thu Sep 15 12:10:19 2022] GET /identity/v3/roles/reader => generated 84 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:10:19.343551 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100f9158-b4f6-41fa-9e14-c00681720068 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:10:19.485724 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100f9158-b4f6-41fa-9e14-c00681720068 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:10:19.489164 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4807f58-2869-47fc-9598-4702b033fca1 None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:10:19.518808 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:10:19.525938 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672589b9-7fda-416c-a08f-8b54c37ba0d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:10:19.526580 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672589b9-7fda-416c-a08f-8b54c37ba0d3 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:19.526860 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672589b9-7fda-416c-a08f-8b54c37ba0d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:19.527132 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672589b9-7fda-416c-a08f-8b54c37ba0d3 None admin] PATH_INFO: `/v3/roles` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:19.527809 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672589b9-7fda-416c-a08f-8b54c37ba0d3 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:10:19.528111 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672589b9-7fda-416c-a08f-8b54c37ba0d3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:10:19.529549 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672589b9-7fda-416c-a08f-8b54c37ba0d3 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:10:19.548888 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 60/116] 10.222.0.15 () {56 vars in 1101 bytes} [Thu Sep 15 12:10:19 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:10:19.549793 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672589b9-7fda-416c-a08f-8b54c37ba0d3 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:10:19.581080 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:10:19.652838 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4807f58-2869-47fc-9598-4702b033fca1 None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:10:19.692707 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235e338f-d60a-4b2b-9768-a5686e61fe63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:10:19.693385 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235e338f-d60a-4b2b-9768-a5686e61fe63 None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:19.693665 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235e338f-d60a-4b2b-9768-a5686e61fe63 None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:19.693936 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235e338f-d60a-4b2b-9768-a5686e61fe63 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:19.695148 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672589b9-7fda-416c-a08f-8b54c37ba0d3 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:10:19.705093 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235e338f-d60a-4b2b-9768-a5686e61fe63 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 12:10:19.710867 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application [None req-235e338f-d60a-4b2b-9768-a5686e61fe63 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 15 12:10:19.710867 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:10:19.710867 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:10:19.710867 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:10:19.710867 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:10:19.710867 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:10:19.710867 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:10:19.710867 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:10:19.710867 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:10:19.710867 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:10:19.710867 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:10:19.710867 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:10:19.710867 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 12:10:19.710867 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 12:10:19.710867 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 12:10:19.710867 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 12:10:19.711700 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 12:10:19.711700 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 12:10:19.711700 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 12:10:19.711700 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 12:10:19.711700 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:10:19.711700 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:10:19.711700 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:10:19.711700 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:10:19.711700 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:10:19.711700 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:10:19.711700 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:10:19.711700 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application with Lock( Sep 15 12:10:19.711700 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:10:19.711700 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:10:19.711700 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:10:19.711700 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:10:19.712947 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:10:19.712947 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:10:19.712947 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:10:19.712947 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:10:19.712947 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 12:10:19.712947 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 12:10:19.712947 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 12:10:19.712947 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 12:10:19.712947 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 12:10:19.712947 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 12:10:19.712947 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 15 12:10:19.712947 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application Sep 15 12:10:19.712947 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 57/117] 10.222.0.15 () {56 vars in 1114 bytes} [Thu Sep 15 12:10:19 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:10:19.713775 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235e338f-d60a-4b2b-9768-a5686e61fe63 None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:10:19.740804 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bebcaa9-ced9-4efc-82f9-f322eed4cdc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:10:19.742019 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bebcaa9-ced9-4efc-82f9-f322eed4cdc0 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:19.742299 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bebcaa9-ced9-4efc-82f9-f322eed4cdc0 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:19.742629 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bebcaa9-ced9-4efc-82f9-f322eed4cdc0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:19.758348 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bebcaa9-ced9-4efc-82f9-f322eed4cdc0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 12:10:19.764536 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application [None req-1bebcaa9-ced9-4efc-82f9-f322eed4cdc0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 15 12:10:19.764536 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:10:19.764536 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:10:19.764536 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:10:19.764536 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:10:19.764536 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:10:19.764536 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:10:19.764536 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:10:19.764536 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:10:19.764536 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:10:19.764536 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:10:19.764536 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:10:19.764536 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 12:10:19.764536 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 12:10:19.764536 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 12:10:19.764536 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 12:10:19.766743 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 12:10:19.766743 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 12:10:19.766743 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 12:10:19.766743 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 12:10:19.766743 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:10:19.766743 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:10:19.766743 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:10:19.766743 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:10:19.766743 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:10:19.766743 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:10:19.766743 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:10:19.766743 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application with Lock( Sep 15 12:10:19.766743 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:10:19.766743 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:10:19.766743 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:10:19.766743 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:10:19.768778 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:10:19.768778 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:10:19.768778 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:10:19.768778 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:10:19.768778 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 12:10:19.768778 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 12:10:19.768778 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 12:10:19.768778 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 12:10:19.768778 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 12:10:19.768778 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 12:10:19.768778 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 15 12:10:19.768778 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 15 12:10:19.768778 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 61/118] 10.222.0.15 () {56 vars in 1096 bytes} [Thu Sep 15 12:10:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:10:19.768778 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bebcaa9-ced9-4efc-82f9-f322eed4cdc0 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:10:19.831186 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:10:19.840431 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235e338f-d60a-4b2b-9768-a5686e61fe63 None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:10:19.848804 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:10:19.882727 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2af14c2-10f5-439f-a58b-d82d53ce9aeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:10:19.883369 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2af14c2-10f5-439f-a58b-d82d53ce9aeb None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:19.883647 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2af14c2-10f5-439f-a58b-d82d53ce9aeb None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:19.883919 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2af14c2-10f5-439f-a58b-d82d53ce9aeb None admin] PATH_INFO: `/v3/users/5b2339bacc154a4b8cbf92a176a2a583` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:19.904664 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2af14c2-10f5-439f-a58b-d82d53ce9aeb None admin] RBAC: Authorizing `identity:get_user(user_id=5b2339bacc154a4b8cbf92a176a2a583)` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:10:19.905083 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bebcaa9-ced9-4efc-82f9-f322eed4cdc0 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:10:19.906196 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2af14c2-10f5-439f-a58b-d82d53ce9aeb None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:10:19.909345 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 58/119] 10.222.0.15 () {56 vars in 1177 bytes} [Thu Sep 15 12:10:19 2022] GET /identity/v3/users/5b2339bacc154a4b8cbf92a176a2a583 => generated 289 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:10:19.909994 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2af14c2-10f5-439f-a58b-d82d53ce9aeb None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:10:19.944075 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c176da-53ea-4a26-866f-cd629e27e2e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:10:19.944724 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c176da-53ea-4a26-866f-cd629e27e2e1 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:19.945038 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c176da-53ea-4a26-866f-cd629e27e2e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:19.945296 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c176da-53ea-4a26-866f-cd629e27e2e1 None admin] PATH_INFO: `/v3/roles` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:19.945960 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c176da-53ea-4a26-866f-cd629e27e2e1 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:10:19.946259 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c176da-53ea-4a26-866f-cd629e27e2e1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:10:19.947819 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c176da-53ea-4a26-866f-cd629e27e2e1 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:10:19.963512 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 62/120] 10.222.0.15 () {56 vars in 1111 bytes} [Thu Sep 15 12:10:19 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:10:19.964784 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41ff9cf-ba3f-40a7-b6aa-61bfb6b7616c None None] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:19.965076 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41ff9cf-ba3f-40a7-b6aa-61bfb6b7616c None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:19.965348 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41ff9cf-ba3f-40a7-b6aa-61bfb6b7616c None None] PATH_INFO: `/` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:19.966846 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 63/121] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 12:10:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:10:19.967561 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41ff9cf-ba3f-40a7-b6aa-61bfb6b7616c None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:10:20.046288 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2af14c2-10f5-439f-a58b-d82d53ce9aeb None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:10:20.082734 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cccec31-8e30-4148-9eba-e9e2f827499b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:10:20.083365 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cccec31-8e30-4148-9eba-e9e2f827499b None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:20.083649 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cccec31-8e30-4148-9eba-e9e2f827499b None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:20.083931 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cccec31-8e30-4148-9eba-e9e2f827499b None admin] PATH_INFO: `/v3/roles` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:20.085047 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cccec31-8e30-4148-9eba-e9e2f827499b None admin] RBAC: Adding query filter params (name=admin) {{(pid=81449) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:10:20.085362 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cccec31-8e30-4148-9eba-e9e2f827499b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:10:20.086752 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cccec31-8e30-4148-9eba-e9e2f827499b None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:10:20.098682 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 59/122] 10.222.0.15 () {56 vars in 1099 bytes} [Thu Sep 15 12:10:19 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:10:20.100015 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cccec31-8e30-4148-9eba-e9e2f827499b None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:10:20.105058 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41ff9cf-ba3f-40a7-b6aa-61bfb6b7616c None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:10:20.128146 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:10:20.145476 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:10:20.155778 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eeea21f-0178-420c-99b8-e4ab5bd27560 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:10:20.156573 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eeea21f-0178-420c-99b8-e4ab5bd27560 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:20.156947 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eeea21f-0178-420c-99b8-e4ab5bd27560 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:20.157289 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eeea21f-0178-420c-99b8-e4ab5bd27560 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:20.239357 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eeea21f-0178-420c-99b8-e4ab5bd27560 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:10:20.240162 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cccec31-8e30-4148-9eba-e9e2f827499b None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:10:20.241057 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eeea21f-0178-420c-99b8-e4ab5bd27560 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:10:20.283450 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a5ecb4-f111-4556-b595-690f18668013 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:10:20.284090 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a5ecb4-f111-4556-b595-690f18668013 None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:20.284375 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a5ecb4-f111-4556-b595-690f18668013 None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:20.284648 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a5ecb4-f111-4556-b595-690f18668013 None admin] PATH_INFO: `/v3/users/6147f6de110b41cd99def2cbeb937ad1` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:20.285829 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a5ecb4-f111-4556-b595-690f18668013 None admin] RBAC: Authorizing `identity:get_user(user_id=6147f6de110b41cd99def2cbeb937ad1)` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:10:20.287798 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a5ecb4-f111-4556-b595-690f18668013 None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:10:20.290431 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 60/123] 10.222.0.15 () {56 vars in 1177 bytes} [Thu Sep 15 12:10:20 2022] GET /identity/v3/users/6147f6de110b41cd99def2cbeb937ad1 => generated 275 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:10:20.291578 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fbb38c-61e0-4e95-99b1-1e9fb2e03a9f None None] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:20.291864 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fbb38c-61e0-4e95-99b1-1e9fb2e03a9f None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:20.292173 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fbb38c-61e0-4e95-99b1-1e9fb2e03a9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:20.317483 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.auth.core [None req-b4fbb38c-61e0-4e95-99b1-1e9fb2e03a9f None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81449) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:10:20.319280 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-b4fbb38c-61e0-4e95-99b1-1e9fb2e03a9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:10:20.362247 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 64/124] 10.222.0.15 () {58 vars in 1282 bytes} [Thu Sep 15 12:10:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 15 12:10:20.363606 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eeea21f-0178-420c-99b8-e4ab5bd27560 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:10:20.384007 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:10:20.412331 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 61/125] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 12:10:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 12:10:20.413786 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fbb38c-61e0-4e95-99b1-1e9fb2e03a9f None None] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:10:20.453434 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:10:20.480173 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eeea21f-0178-420c-99b8-e4ab5bd27560 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:10:20.517326 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a87795-d08b-41c9-af7b-0300426e38c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:10:20.517978 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a87795-d08b-41c9-af7b-0300426e38c5 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:20.518263 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a87795-d08b-41c9-af7b-0300426e38c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:20.518539 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a87795-d08b-41c9-af7b-0300426e38c5 None admin] PATH_INFO: `/v3/users/c66eff9fe7124011868989f16846a501` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:20.520417 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a87795-d08b-41c9-af7b-0300426e38c5 None admin] RBAC: Authorizing `identity:get_user(user_id=c66eff9fe7124011868989f16846a501)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:10:20.521913 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a87795-d08b-41c9-af7b-0300426e38c5 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:10:20.524074 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 65/126] 10.222.0.15 () {56 vars in 1177 bytes} [Thu Sep 15 12:10:20 2022] GET /identity/v3/users/c66eff9fe7124011868989f16846a501 => generated 247 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:10:20.524867 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a87795-d08b-41c9-af7b-0300426e38c5 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:10:20.600148 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fbb38c-61e0-4e95-99b1-1e9fb2e03a9f None None] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:10:20.665359 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3efc45a-4de5-4705-b5f3-dec4568f76e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:10:20.666384 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3efc45a-4de5-4705-b5f3-dec4568f76e9 None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:20.666863 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3efc45a-4de5-4705-b5f3-dec4568f76e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:20.667560 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3efc45a-4de5-4705-b5f3-dec4568f76e9 None admin] PATH_INFO: `/v3/projects/10a21ffb0b8d4800a805c76f171c40c1` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:20.671961 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3efc45a-4de5-4705-b5f3-dec4568f76e9 None admin] RBAC: Authorizing `identity:get_project(project_id=10a21ffb0b8d4800a805c76f171c40c1)` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:10:20.673656 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3efc45a-4de5-4705-b5f3-dec4568f76e9 None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:10:20.675494 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 62/127] 10.222.0.15 () {56 vars in 1186 bytes} [Thu Sep 15 12:10:20 2022] GET /identity/v3/projects/10a21ffb0b8d4800a805c76f171c40c1 => generated 298 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:10:20.677409 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3efc45a-4de5-4705-b5f3-dec4568f76e9 None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:10:20.703072 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:10:20.733082 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a87795-d08b-41c9-af7b-0300426e38c5 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:10:20.792895 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7d3f64-a025-4f40-8f5a-02ff7c41b120 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:10:20.793554 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7d3f64-a025-4f40-8f5a-02ff7c41b120 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:20.793840 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7d3f64-a025-4f40-8f5a-02ff7c41b120 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:20.794119 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7d3f64-a025-4f40-8f5a-02ff7c41b120 None admin] PATH_INFO: `/v3/projects/10a21ffb0b8d4800a805c76f171c40c1` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:20.795258 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7d3f64-a025-4f40-8f5a-02ff7c41b120 None admin] RBAC: Authorizing `identity:get_project(project_id=10a21ffb0b8d4800a805c76f171c40c1)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:10:20.796858 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7d3f64-a025-4f40-8f5a-02ff7c41b120 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:10:20.798669 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 66/128] 10.222.0.15 () {56 vars in 1186 bytes} [Thu Sep 15 12:10:20 2022] GET /identity/v3/projects/10a21ffb0b8d4800a805c76f171c40c1 => generated 298 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:10:20.799420 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7d3f64-a025-4f40-8f5a-02ff7c41b120 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:10:20.824801 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:10:20.838248 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3efc45a-4de5-4705-b5f3-dec4568f76e9 None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:10:20.878783 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53f61e4-87c7-46da-aa7b-393aeadf10ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:10:20.880282 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53f61e4-87c7-46da-aa7b-393aeadf10ff None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:20.880825 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53f61e4-87c7-46da-aa7b-393aeadf10ff None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:20.881315 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53f61e4-87c7-46da-aa7b-393aeadf10ff None admin] PATH_INFO: `/v3/projects/10a21ffb0b8d4800a805c76f171c40c1` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:20.882827 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53f61e4-87c7-46da-aa7b-393aeadf10ff None admin] RBAC: Authorizing `identity:get_project(project_id=10a21ffb0b8d4800a805c76f171c40c1)` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:10:20.885131 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53f61e4-87c7-46da-aa7b-393aeadf10ff None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:10:20.888255 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 63/129] 10.222.0.15 () {56 vars in 1186 bytes} [Thu Sep 15 12:10:20 2022] GET /identity/v3/projects/10a21ffb0b8d4800a805c76f171c40c1 => generated 298 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:10:20.889302 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53f61e4-87c7-46da-aa7b-393aeadf10ff None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:10:20.922152 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7d3f64-a025-4f40-8f5a-02ff7c41b120 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:10:20.965775 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5931ab8b-35b9-4d8a-a58f-a91db03cce94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:10:20.966472 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5931ab8b-35b9-4d8a-a58f-a91db03cce94 None admin] REQUEST_METHOD: `PUT` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:20.966742 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5931ab8b-35b9-4d8a-a58f-a91db03cce94 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:20.967025 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5931ab8b-35b9-4d8a-a58f-a91db03cce94 None admin] PATH_INFO: `/v3/projects/10a21ffb0b8d4800a805c76f171c40c1/users/6147f6de110b41cd99def2cbeb937ad1/roles/1081b93ddfd547649ba8582f7a6e719e` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:20.969433 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5931ab8b-35b9-4d8a-a58f-a91db03cce94 None admin] RBAC: Authorizing `identity:create_grant(project_id=10a21ffb0b8d4800a805c76f171c40c1, user_id=6147f6de110b41cd99def2cbeb937ad1, role_id=1081b93ddfd547649ba8582f7a6e719e)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:10:20.971583 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5931ab8b-35b9-4d8a-a58f-a91db03cce94 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:10:20.996022 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 67/130] 10.222.0.15 () {58 vars in 1439 bytes} [Thu Sep 15 12:10:20 2022] PUT /identity/v3/projects/10a21ffb0b8d4800a805c76f171c40c1/users/6147f6de110b41cd99def2cbeb937ad1/roles/1081b93ddfd547649ba8582f7a6e719e => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:10:20.997467 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5931ab8b-35b9-4d8a-a58f-a91db03cce94 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:10:21.045146 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:10:21.048098 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53f61e4-87c7-46da-aa7b-393aeadf10ff None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:10:21.124410 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4270d8-3993-4a25-b186-488ca2624ac9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:10:21.125080 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4270d8-3993-4a25-b186-488ca2624ac9 None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:21.125376 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4270d8-3993-4a25-b186-488ca2624ac9 None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:21.125632 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4270d8-3993-4a25-b186-488ca2624ac9 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:21.150050 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c4270d8-3993-4a25-b186-488ca2624ac9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 12:10:21.154999 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application [None req-8c4270d8-3993-4a25-b186-488ca2624ac9 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 15 12:10:21.154999 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:10:21.154999 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:10:21.154999 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:10:21.154999 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:10:21.154999 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:10:21.154999 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:10:21.154999 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:10:21.154999 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:10:21.154999 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:10:21.154999 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:10:21.154999 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:10:21.154999 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 12:10:21.154999 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 12:10:21.154999 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 12:10:21.154999 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 12:10:21.155933 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 12:10:21.155933 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 12:10:21.155933 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 12:10:21.155933 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 12:10:21.155933 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:10:21.155933 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:10:21.155933 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:10:21.155933 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:10:21.155933 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:10:21.155933 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:10:21.155933 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:10:21.155933 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application with Lock( Sep 15 12:10:21.155933 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:10:21.155933 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:10:21.155933 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:10:21.155933 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:10:21.156702 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:10:21.156702 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:10:21.156702 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:10:21.156702 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:10:21.156702 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 12:10:21.156702 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 12:10:21.156702 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 12:10:21.156702 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 12:10:21.156702 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 12:10:21.156702 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 12:10:21.156702 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 15 12:10:21.156702 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application Sep 15 12:10:21.156702 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 64/131] 10.222.0.15 () {56 vars in 1099 bytes} [Thu Sep 15 12:10:20 2022] GET /identity/v3/roles/member => generated 84 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:10:21.158276 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4270d8-3993-4a25-b186-488ca2624ac9 None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:10:21.197792 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:10:21.212868 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5931ab8b-35b9-4d8a-a58f-a91db03cce94 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:10:21.249939 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ffa1b80-2071-49a1-950a-204fe069e7a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:10:21.250566 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffa1b80-2071-49a1-950a-204fe069e7a8 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:21.250851 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffa1b80-2071-49a1-950a-204fe069e7a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:21.251125 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ffa1b80-2071-49a1-950a-204fe069e7a8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:21.252574 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ffa1b80-2071-49a1-950a-204fe069e7a8 None admin] RBAC: Adding query filter params (role.id=1807da0df67b49739d033f8c0e7456f0, scope.project.id=10a21ffb0b8d4800a805c76f171c40c1, user.id=5b2339bacc154a4b8cbf92a176a2a583) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:10:21.252929 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ffa1b80-2071-49a1-950a-204fe069e7a8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:10:21.254458 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ffa1b80-2071-49a1-950a-204fe069e7a8 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:10:21.260428 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 68/132] 10.222.0.15 () {56 vars in 1374 bytes} [Thu Sep 15 12:10:20 2022] GET /identity/v3/role_assignments?user.id=5b2339bacc154a4b8cbf92a176a2a583&scope.project.id=10a21ffb0b8d4800a805c76f171c40c1&role.id=1807da0df67b49739d033f8c0e7456f0 => generated 259 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:10:21.261378 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ffa1b80-2071-49a1-950a-204fe069e7a8 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:10:21.314653 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4270d8-3993-4a25-b186-488ca2624ac9 None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:10:21.381639 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562ada5d-0cfd-4261-83d0-253cba524c4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:10:21.383002 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562ada5d-0cfd-4261-83d0-253cba524c4f None admin] REQUEST_METHOD: `PUT` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:21.383290 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562ada5d-0cfd-4261-83d0-253cba524c4f None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:21.383682 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562ada5d-0cfd-4261-83d0-253cba524c4f None admin] PATH_INFO: `/v3/projects/10a21ffb0b8d4800a805c76f171c40c1/users/c66eff9fe7124011868989f16846a501/roles/9c769b5c82884766b2c510b1f0f1364c` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:21.385685 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562ada5d-0cfd-4261-83d0-253cba524c4f None admin] RBAC: Authorizing `identity:create_grant(project_id=10a21ffb0b8d4800a805c76f171c40c1, user_id=c66eff9fe7124011868989f16846a501, role_id=9c769b5c82884766b2c510b1f0f1364c)` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:10:21.387177 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562ada5d-0cfd-4261-83d0-253cba524c4f None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:10:21.395633 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:10:21.424037 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 65/133] 10.222.0.15 () {58 vars in 1439 bytes} [Thu Sep 15 12:10:21 2022] PUT /identity/v3/projects/10a21ffb0b8d4800a805c76f171c40c1/users/c66eff9fe7124011868989f16846a501/roles/9c769b5c82884766b2c510b1f0f1364c => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:10:21.446490 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ffa1b80-2071-49a1-950a-204fe069e7a8 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:10:21.506799 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d33ee2-122c-403a-9f4d-39441c322dc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:10:21.508014 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d33ee2-122c-403a-9f4d-39441c322dc1 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:21.508300 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d33ee2-122c-403a-9f4d-39441c322dc1 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:21.508582 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d33ee2-122c-403a-9f4d-39441c322dc1 None admin] PATH_INFO: `/v3/roles` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:21.509313 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d33ee2-122c-403a-9f4d-39441c322dc1 None admin] RBAC: Adding query filter params (name=member) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:10:21.509621 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d33ee2-122c-403a-9f4d-39441c322dc1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:10:21.511076 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19d33ee2-122c-403a-9f4d-39441c322dc1 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:10:21.525003 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 69/134] 10.222.0.15 () {56 vars in 1101 bytes} [Thu Sep 15 12:10:21 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:10:21.531002 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562ada5d-0cfd-4261-83d0-253cba524c4f None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:10:21.637697 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:10:21.661663 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562ada5d-0cfd-4261-83d0-253cba524c4f None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:10:21.709353 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c35de4-0126-409c-a78c-d3644f76be3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:10:21.710046 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c35de4-0126-409c-a78c-d3644f76be3d None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:21.710996 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c35de4-0126-409c-a78c-d3644f76be3d None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:21.711320 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c35de4-0126-409c-a78c-d3644f76be3d None admin] PATH_INFO: `/v3/users/6147f6de110b41cd99def2cbeb937ad1` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:21.712666 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c35de4-0126-409c-a78c-d3644f76be3d None admin] RBAC: Authorizing `identity:get_user(user_id=6147f6de110b41cd99def2cbeb937ad1)` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:10:21.714319 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c35de4-0126-409c-a78c-d3644f76be3d None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:10:21.716356 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 66/135] 10.222.0.15 () {56 vars in 1177 bytes} [Thu Sep 15 12:10:21 2022] GET /identity/v3/users/6147f6de110b41cd99def2cbeb937ad1 => generated 275 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:10:21.721710 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d33ee2-122c-403a-9f4d-39441c322dc1 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:10:21.755480 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0aa2d9c-50f6-4a04-ad96-9fcd6bfab257 None None] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:21.755787 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0aa2d9c-50f6-4a04-ad96-9fcd6bfab257 None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:21.756051 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0aa2d9c-50f6-4a04-ad96-9fcd6bfab257 None None] PATH_INFO: `/` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:21.758039 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 67/136] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 12:10:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:10:21.769022 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fc4863-5803-4ed4-b20f-83045f6adaf3 None None] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:21.769316 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fc4863-5803-4ed4-b20f-83045f6adaf3 None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:21.769587 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fc4863-5803-4ed4-b20f-83045f6adaf3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:21.801801 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.auth.core [None req-21fc4863-5803-4ed4-b20f-83045f6adaf3 None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81449) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:10:21.803031 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-21fc4863-5803-4ed4-b20f-83045f6adaf3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:10:21.852105 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d33ee2-122c-403a-9f4d-39441c322dc1 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:10:21.901361 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a55222-e206-4f82-9427-316bd1dc16e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:10:21.902900 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a55222-e206-4f82-9427-316bd1dc16e7 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:21.903391 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a55222-e206-4f82-9427-316bd1dc16e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:21.904151 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a55222-e206-4f82-9427-316bd1dc16e7 None admin] PATH_INFO: `/v3/projects/53dad4f5e490445fa10339bdf7c51660` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:21.908381 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a55222-e206-4f82-9427-316bd1dc16e7 None admin] RBAC: Authorizing `identity:get_project(project_id=53dad4f5e490445fa10339bdf7c51660)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:10:21.910372 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a55222-e206-4f82-9427-316bd1dc16e7 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:10:21.912257 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 70/137] 10.222.0.15 () {56 vars in 1186 bytes} [Thu Sep 15 12:10:21 2022] GET /identity/v3/projects/53dad4f5e490445fa10339bdf7c51660 => generated 312 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:10:21.921781 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a55222-e206-4f82-9427-316bd1dc16e7 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:10:21.933133 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:10:21.949801 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 68/138] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 12:10:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 15 12:10:22.068992 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edca1c3f-b319-4d5e-b10e-45d4fd6d3638 None None] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:22.069345 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edca1c3f-b319-4d5e-b10e-45d4fd6d3638 None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:22.069588 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edca1c3f-b319-4d5e-b10e-45d4fd6d3638 None None] PATH_INFO: `/` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:22.071075 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 69/139] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 12:10:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 15 12:10:22.076090 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8674e9-c94f-45b7-946b-ff12ce04ac57 None None] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:22.076376 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8674e9-c94f-45b7-946b-ff12ce04ac57 None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:22.076654 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8674e9-c94f-45b7-946b-ff12ce04ac57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:22.095988 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.auth.core [None req-7a8674e9-c94f-45b7-946b-ff12ce04ac57 None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81449) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:10:22.097296 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-7a8674e9-c94f-45b7-946b-ff12ce04ac57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:10:22.105590 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a55222-e206-4f82-9427-316bd1dc16e7 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:10:22.173603 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c155f016-c732-45cc-a4aa-4661e8ff678f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:10:22.174636 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c155f016-c732-45cc-a4aa-4661e8ff678f None admin] REQUEST_METHOD: `PUT` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:22.176089 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c155f016-c732-45cc-a4aa-4661e8ff678f None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:22.176592 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c155f016-c732-45cc-a4aa-4661e8ff678f None admin] PATH_INFO: `/v3/projects/53dad4f5e490445fa10339bdf7c51660/users/6147f6de110b41cd99def2cbeb937ad1/roles/0679d6628a25438d90dc4bdc0401eda2` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:22.185532 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c155f016-c732-45cc-a4aa-4661e8ff678f None admin] RBAC: Authorizing `identity:create_grant(project_id=53dad4f5e490445fa10339bdf7c51660, user_id=6147f6de110b41cd99def2cbeb937ad1, role_id=0679d6628a25438d90dc4bdc0401eda2)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:10:22.187448 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c155f016-c732-45cc-a4aa-4661e8ff678f None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:10:22.214058 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 71/140] 10.222.0.15 () {58 vars in 1439 bytes} [Thu Sep 15 12:10:21 2022] PUT /identity/v3/projects/53dad4f5e490445fa10339bdf7c51660/users/6147f6de110b41cd99def2cbeb937ad1/roles/0679d6628a25438d90dc4bdc0401eda2 => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:10:22.230778 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 70/141] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 12:10:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 15 12:10:22.354427 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c155f016-c732-45cc-a4aa-4661e8ff678f None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:10:22.453428 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:10:22.503632 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c155f016-c732-45cc-a4aa-4661e8ff678f None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:10:22.543348 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835b8654-53e6-48c9-ad89-4a57193f4b1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:10:22.544056 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835b8654-53e6-48c9-ad89-4a57193f4b1a None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:22.544346 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835b8654-53e6-48c9-ad89-4a57193f4b1a None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:22.544624 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835b8654-53e6-48c9-ad89-4a57193f4b1a None admin] PATH_INFO: `/v3/roles/member` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:22.561627 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-835b8654-53e6-48c9-ad89-4a57193f4b1a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 12:10:22.567012 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application [None req-835b8654-53e6-48c9-ad89-4a57193f4b1a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 15 12:10:22.567012 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:10:22.567012 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:10:22.567012 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:10:22.567012 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:10:22.567012 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:10:22.567012 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:10:22.567012 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:10:22.567012 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:10:22.567012 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:10:22.567012 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:10:22.567012 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:10:22.567012 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 12:10:22.567012 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 12:10:22.567012 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 12:10:22.567012 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 12:10:22.567879 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 12:10:22.567879 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 12:10:22.567879 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 12:10:22.567879 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 12:10:22.567879 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:10:22.567879 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:10:22.567879 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:10:22.567879 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:10:22.567879 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:10:22.567879 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:10:22.567879 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:10:22.567879 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application with Lock( Sep 15 12:10:22.567879 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:10:22.567879 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:10:22.567879 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:10:22.567879 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:10:22.568651 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:10:22.568651 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:10:22.568651 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:10:22.568651 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:10:22.568651 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 12:10:22.568651 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 12:10:22.568651 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 12:10:22.568651 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 12:10:22.568651 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 12:10:22.568651 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 12:10:22.568651 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 15 12:10:22.568651 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 15 12:10:22.568651 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 72/142] 10.222.0.15 () {56 vars in 1099 bytes} [Thu Sep 15 12:10:22 2022] GET /identity/v3/roles/member => generated 84 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:10:22.577883 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8674e9-c94f-45b7-946b-ff12ce04ac57 None None] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:10:22.588662 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:10:22.707322 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8674e9-c94f-45b7-946b-ff12ce04ac57 None None] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:10:22.753527 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b240d17e-ba02-498b-961c-58f6970eae56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:10:22.754213 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b240d17e-ba02-498b-961c-58f6970eae56 None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:22.754440 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b240d17e-ba02-498b-961c-58f6970eae56 None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:22.754716 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b240d17e-ba02-498b-961c-58f6970eae56 None admin] PATH_INFO: `/v3/roles` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:22.755388 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b240d17e-ba02-498b-961c-58f6970eae56 None admin] RBAC: Adding query filter params (name=member) {{(pid=81449) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:10:22.755703 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b240d17e-ba02-498b-961c-58f6970eae56 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:10:22.757775 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b240d17e-ba02-498b-961c-58f6970eae56 None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:10:22.772394 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 71/143] 10.222.0.15 () {56 vars in 1101 bytes} [Thu Sep 15 12:10:22 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:10:22.778160 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835b8654-53e6-48c9-ad89-4a57193f4b1a None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:10:22.894254 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835b8654-53e6-48c9-ad89-4a57193f4b1a None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:10:22.926218 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:10:22.930067 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b240d17e-ba02-498b-961c-58f6970eae56 None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:10:22.950946 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc88d24-b071-4f39-adf0-dd5da9047a1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:10:22.951843 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc88d24-b071-4f39-adf0-dd5da9047a1c None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:22.952123 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc88d24-b071-4f39-adf0-dd5da9047a1c None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:22.952395 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc88d24-b071-4f39-adf0-dd5da9047a1c None admin] PATH_INFO: `/v3/users/6147f6de110b41cd99def2cbeb937ad1` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:22.953565 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc88d24-b071-4f39-adf0-dd5da9047a1c None admin] RBAC: Authorizing `identity:get_user(user_id=6147f6de110b41cd99def2cbeb937ad1)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:10:22.955013 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc88d24-b071-4f39-adf0-dd5da9047a1c None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:10:22.956790 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 73/144] 10.222.0.15 () {56 vars in 1177 bytes} [Thu Sep 15 12:10:22 2022] GET /identity/v3/users/6147f6de110b41cd99def2cbeb937ad1 => generated 275 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:10:22.962417 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc88d24-b071-4f39-adf0-dd5da9047a1c None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:10:23.069241 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:10:23.096231 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b240d17e-ba02-498b-961c-58f6970eae56 None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:10:23.140415 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a70779c-ac86-44b0-b759-b331dae9fad0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:10:23.141469 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a70779c-ac86-44b0-b759-b331dae9fad0 None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:23.141976 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a70779c-ac86-44b0-b759-b331dae9fad0 None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:23.142436 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a70779c-ac86-44b0-b759-b331dae9fad0 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:23.149514 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a70779c-ac86-44b0-b759-b331dae9fad0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:10:23.151256 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a70779c-ac86-44b0-b759-b331dae9fad0 None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:10:23.153095 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 72/145] 10.222.0.15 () {56 vars in 1108 bytes} [Thu Sep 15 12:10:22 2022] GET /identity/v3/domains/default => generated 199 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 12:10:23.154957 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc88d24-b071-4f39-adf0-dd5da9047a1c None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:10:23.170040 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a70779c-ac86-44b0-b759-b331dae9fad0 None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:10:23.220784 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4561973-fdb8-4036-8a7f-1d2380d6e01d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:10:23.221425 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4561973-fdb8-4036-8a7f-1d2380d6e01d None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:23.221700 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4561973-fdb8-4036-8a7f-1d2380d6e01d None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:23.221972 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4561973-fdb8-4036-8a7f-1d2380d6e01d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:23.322515 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a70779c-ac86-44b0-b759-b331dae9fad0 None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:10:23.374381 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd81d056-79aa-4fdb-a78f-7fa5a55709bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:10:23.374778 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:10:23.374996 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd81d056-79aa-4fdb-a78f-7fa5a55709bc None admin] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:23.375278 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd81d056-79aa-4fdb-a78f-7fa5a55709bc None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:23.375548 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd81d056-79aa-4fdb-a78f-7fa5a55709bc None admin] PATH_INFO: `/v3/users` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:23.376198 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd81d056-79aa-4fdb-a78f-7fa5a55709bc None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:10:23.377672 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd81d056-79aa-4fdb-a78f-7fa5a55709bc None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:10:23.395680 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:10:23.398317 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4561973-fdb8-4036-8a7f-1d2380d6e01d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:10:23.399933 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4561973-fdb8-4036-8a7f-1d2380d6e01d None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:10:23.420083 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 73/146] 10.222.0.15 () {60 vars in 1132 bytes} [Thu Sep 15 12:10:23 2022] POST /identity/v3/users => generated 283 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 12:10:23.556427 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 74/147] 10.222.0.15 () {58 vars in 1282 bytes} [Thu Sep 15 12:10:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 589 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 15 12:10:23.558061 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd81d056-79aa-4fdb-a78f-7fa5a55709bc None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:10:23.698382 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd81d056-79aa-4fdb-a78f-7fa5a55709bc None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:10:23.745713 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938fb504-e444-4469-b6b1-bd1a815b0f83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:10:23.746366 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938fb504-e444-4469-b6b1-bd1a815b0f83 None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:23.746647 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938fb504-e444-4469-b6b1-bd1a815b0f83 None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:23.746919 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938fb504-e444-4469-b6b1-bd1a815b0f83 None admin] PATH_INFO: `/v3/projects/10a21ffb0b8d4800a805c76f171c40c1` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:23.748068 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938fb504-e444-4469-b6b1-bd1a815b0f83 None admin] RBAC: Authorizing `identity:get_project(project_id=10a21ffb0b8d4800a805c76f171c40c1)` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:10:23.749539 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938fb504-e444-4469-b6b1-bd1a815b0f83 None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:10:23.751320 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 74/148] 10.222.0.15 () {56 vars in 1186 bytes} [Thu Sep 15 12:10:23 2022] GET /identity/v3/projects/10a21ffb0b8d4800a805c76f171c40c1 => generated 298 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:10:23.757884 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4561973-fdb8-4036-8a7f-1d2380d6e01d None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:10:23.900829 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:10:23.908148 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4561973-fdb8-4036-8a7f-1d2380d6e01d None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:10:23.958021 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78aa2c34-b311-448f-88f8-8992d3461ea8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:10:23.958562 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78aa2c34-b311-448f-88f8-8992d3461ea8 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:23.958839 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78aa2c34-b311-448f-88f8-8992d3461ea8 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:23.959113 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78aa2c34-b311-448f-88f8-8992d3461ea8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:23.960781 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78aa2c34-b311-448f-88f8-8992d3461ea8 None admin] RBAC: Adding query filter params (role.id=0679d6628a25438d90dc4bdc0401eda2, scope.project.id=10a21ffb0b8d4800a805c76f171c40c1, user.id=6147f6de110b41cd99def2cbeb937ad1) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:10:23.960781 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78aa2c34-b311-448f-88f8-8992d3461ea8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:10:23.963306 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78aa2c34-b311-448f-88f8-8992d3461ea8 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:10:23.973379 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 75/149] 10.222.0.15 () {56 vars in 1374 bytes} [Thu Sep 15 12:10:23 2022] GET /identity/v3/role_assignments?user.id=6147f6de110b41cd99def2cbeb937ad1&scope.project.id=10a21ffb0b8d4800a805c76f171c40c1&role.id=0679d6628a25438d90dc4bdc0401eda2 => generated 608 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:10:24.425537 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1237c163-3b26-4f82-9618-e1c9edefee63 None None] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:24.426020 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1237c163-3b26-4f82-9618-e1c9edefee63 None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:24.426020 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1237c163-3b26-4f82-9618-e1c9edefee63 None None] PATH_INFO: `/` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:24.427335 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 75/150] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 12:10:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:10:24.434145 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45cc93c-beb2-491b-8e32-9270ca5e6cd0 None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:24.434591 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45cc93c-beb2-491b-8e32-9270ca5e6cd0 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:24.434793 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e45cc93c-beb2-491b-8e32-9270ca5e6cd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:24.456951 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-e45cc93c-beb2-491b-8e32-9270ca5e6cd0 None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:10:24.458138 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-e45cc93c-beb2-491b-8e32-9270ca5e6cd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:10:24.474136 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff73dbe8-5f67-422d-857e-f48533ba24db None None] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:24.474424 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff73dbe8-5f67-422d-857e-f48533ba24db None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:24.474701 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff73dbe8-5f67-422d-857e-f48533ba24db None None] PATH_INFO: `/` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:24.476613 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 76/151] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 12:10:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:10:24.482587 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3864dfc9-49fc-4ffb-a41c-62a3311454ac None None] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:24.482872 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3864dfc9-49fc-4ffb-a41c-62a3311454ac None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:24.483143 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3864dfc9-49fc-4ffb-a41c-62a3311454ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:24.504417 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.auth.core [None req-3864dfc9-49fc-4ffb-a41c-62a3311454ac None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81449) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:10:24.505640 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-3864dfc9-49fc-4ffb-a41c-62a3311454ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:10:24.572077 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 76/152] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 12:10:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 12:10:24.574478 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:10:24.587742 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 77/153] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 12:10:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 15 12:10:24.870005 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00f837c-72b4-4910-8050-f0820a553c0a None None] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:24.870379 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00f837c-72b4-4910-8050-f0820a553c0a None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:24.870572 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00f837c-72b4-4910-8050-f0820a553c0a None None] PATH_INFO: `/` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:24.872043 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 77/154] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 12:10:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:10:24.877449 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4e7690-05a4-465a-aa76-db31acff3de6 None None] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:24.877963 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4e7690-05a4-465a-aa76-db31acff3de6 None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:24.878511 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4e7690-05a4-465a-aa76-db31acff3de6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:24.905830 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.auth.core [None req-bd4e7690-05a4-465a-aa76-db31acff3de6 None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81449) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:10:24.907333 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-bd4e7690-05a4-465a-aa76-db31acff3de6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:10:25.007039 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 78/155] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 12:10:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 12:10:25.044879 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00f837c-72b4-4910-8050-f0820a553c0a None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:10:25.144254 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:10:25.163770 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00f837c-72b4-4910-8050-f0820a553c0a None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:10:25.209193 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4e7690-05a4-465a-aa76-db31acff3de6 None None] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:10:25.211268 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f82133-5c6c-4b93-8c96-701300ea2a06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:10:25.216791 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f82133-5c6c-4b93-8c96-701300ea2a06 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:25.216791 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f82133-5c6c-4b93-8c96-701300ea2a06 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:25.216791 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f82133-5c6c-4b93-8c96-701300ea2a06 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:25.224507 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f82133-5c6c-4b93-8c96-701300ea2a06 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 12:10:25.230886 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application [None req-e0f82133-5c6c-4b93-8c96-701300ea2a06 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 15 12:10:25.230886 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:10:25.230886 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:10:25.230886 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:10:25.230886 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:10:25.230886 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:10:25.230886 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:10:25.230886 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:10:25.230886 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:10:25.230886 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:10:25.230886 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:10:25.230886 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:10:25.230886 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 12:10:25.230886 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 12:10:25.230886 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 12:10:25.230886 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 12:10:25.231762 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 12:10:25.231762 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 12:10:25.231762 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 12:10:25.231762 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 12:10:25.231762 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:10:25.231762 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:10:25.231762 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:10:25.231762 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:10:25.231762 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:10:25.231762 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:10:25.231762 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:10:25.231762 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application with Lock( Sep 15 12:10:25.231762 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:10:25.231762 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:10:25.231762 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:10:25.231762 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:10:25.232524 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:10:25.232524 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:10:25.232524 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:10:25.232524 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:10:25.232524 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 12:10:25.232524 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 12:10:25.232524 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 12:10:25.232524 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 12:10:25.232524 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 12:10:25.232524 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 12:10:25.232524 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 15 12:10:25.232524 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 15 12:10:25.232524 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 78/156] 10.222.0.15 () {56 vars in 1099 bytes} [Thu Sep 15 12:10:25 2022] GET /identity/v3/roles/reader => generated 84 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:10:25.239469 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f82133-5c6c-4b93-8c96-701300ea2a06 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:10:25.407595 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4e7690-05a4-465a-aa76-db31acff3de6 None None] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:10:25.444690 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f82133-5c6c-4b93-8c96-701300ea2a06 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:10:25.470321 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:10:25.496343 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20abe361-893f-4a83-adc4-5d85b41cb40b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:10:25.497015 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20abe361-893f-4a83-adc4-5d85b41cb40b None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:25.497293 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20abe361-893f-4a83-adc4-5d85b41cb40b None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:25.497564 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20abe361-893f-4a83-adc4-5d85b41cb40b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:25.508205 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20abe361-893f-4a83-adc4-5d85b41cb40b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 12:10:25.513422 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application [None req-20abe361-893f-4a83-adc4-5d85b41cb40b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 15 12:10:25.513422 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:10:25.513422 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:10:25.513422 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:10:25.513422 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:10:25.513422 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:10:25.513422 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:10:25.513422 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:10:25.513422 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:10:25.513422 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:10:25.513422 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:10:25.513422 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:10:25.513422 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 12:10:25.513422 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 12:10:25.513422 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 12:10:25.513422 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 12:10:25.514428 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 12:10:25.514428 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 12:10:25.514428 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 12:10:25.514428 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 12:10:25.514428 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:10:25.514428 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:10:25.514428 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:10:25.514428 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:10:25.514428 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:10:25.514428 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:10:25.514428 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:10:25.514428 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application with Lock( Sep 15 12:10:25.514428 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:10:25.514428 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:10:25.514428 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:10:25.514428 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:10:25.515295 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:10:25.515295 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:10:25.515295 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:10:25.515295 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:10:25.515295 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 12:10:25.515295 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 12:10:25.515295 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 12:10:25.515295 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 12:10:25.515295 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 12:10:25.515295 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 12:10:25.515295 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 15 12:10:25.515295 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application Sep 15 12:10:25.515295 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 79/157] 10.222.0.15 () {56 vars in 1114 bytes} [Thu Sep 15 12:10:25 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 306 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 15 12:10:25.520846 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d23cd4-c172-4310-b020-a306bf9c933c None None] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:25.520846 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d23cd4-c172-4310-b020-a306bf9c933c None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:25.520846 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d23cd4-c172-4310-b020-a306bf9c933c None None] PATH_INFO: `/` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:25.521345 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f273e8d-491f-47b9-81e3-4852077e9227 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:10:25.521345 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f273e8d-491f-47b9-81e3-4852077e9227 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:25.521798 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 80/158] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 12:10:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:10:25.522227 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f273e8d-491f-47b9-81e3-4852077e9227 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:25.522227 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f273e8d-491f-47b9-81e3-4852077e9227 None admin] PATH_INFO: `/v3/roles` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:25.522827 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d23cd4-c172-4310-b020-a306bf9c933c None None] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:10:25.522988 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f273e8d-491f-47b9-81e3-4852077e9227 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:10:25.522988 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f273e8d-491f-47b9-81e3-4852077e9227 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:10:25.524484 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f273e8d-491f-47b9-81e3-4852077e9227 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:10:25.540379 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 79/159] 10.222.0.15 () {56 vars in 1101 bytes} [Thu Sep 15 12:10:25 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:10:25.541649 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6cfcc4-39b5-4352-90d2-7230254e488a None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:25.541931 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6cfcc4-39b5-4352-90d2-7230254e488a None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:25.542204 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6cfcc4-39b5-4352-90d2-7230254e488a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:25.586553 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-0e6cfcc4-39b5-4352-90d2-7230254e488a None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:10:25.587751 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-0e6cfcc4-39b5-4352-90d2-7230254e488a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:10:25.596396 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:10:25.691599 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d23cd4-c172-4310-b020-a306bf9c933c None None] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:10:25.755113 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 80/160] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 12:10:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 12:10:25.756492 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6cfcc4-39b5-4352-90d2-7230254e488a None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:10:25.765457 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d521c9fa-44fc-48b0-b99f-697c51dcc878 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:10:25.766102 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d521c9fa-44fc-48b0-b99f-697c51dcc878 None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:25.766378 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d521c9fa-44fc-48b0-b99f-697c51dcc878 None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:25.766646 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d521c9fa-44fc-48b0-b99f-697c51dcc878 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:25.785683 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d521c9fa-44fc-48b0-b99f-697c51dcc878 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 12:10:25.791141 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application [None req-d521c9fa-44fc-48b0-b99f-697c51dcc878 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 15 12:10:25.791141 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:10:25.791141 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:10:25.791141 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:10:25.791141 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:10:25.791141 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:10:25.791141 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:10:25.791141 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:10:25.791141 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:10:25.791141 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:10:25.791141 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:10:25.791141 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:10:25.791141 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 12:10:25.791141 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 12:10:25.791141 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 12:10:25.791141 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 12:10:25.792115 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 12:10:25.792115 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 12:10:25.792115 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 12:10:25.792115 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 12:10:25.792115 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:10:25.792115 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:10:25.792115 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:10:25.792115 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:10:25.792115 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:10:25.792115 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:10:25.792115 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:10:25.792115 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application with Lock( Sep 15 12:10:25.792115 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:10:25.792115 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:10:25.792115 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:10:25.792115 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:10:25.792968 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:10:25.792968 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:10:25.792968 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:10:25.792968 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:10:25.792968 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 12:10:25.792968 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 12:10:25.792968 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 12:10:25.792968 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 12:10:25.792968 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 12:10:25.792968 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 12:10:25.792968 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 15 12:10:25.792968 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application Sep 15 12:10:25.792968 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 81/161] 10.222.0.15 () {56 vars in 1096 bytes} [Thu Sep 15 12:10:25 2022] GET /identity/v3/roles/admin => generated 83 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:10:25.801111 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d521c9fa-44fc-48b0-b99f-697c51dcc878 None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:10:25.875233 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6cfcc4-39b5-4352-90d2-7230254e488a None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:10:25.886593 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:10:25.918446 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e18300-bf19-4a54-a98a-89c040dc2c80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:10:25.919736 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e18300-bf19-4a54-a98a-89c040dc2c80 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:25.920835 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e18300-bf19-4a54-a98a-89c040dc2c80 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:25.921616 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e18300-bf19-4a54-a98a-89c040dc2c80 None admin] PATH_INFO: `/v3/roles` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:25.922994 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e18300-bf19-4a54-a98a-89c040dc2c80 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:10:25.924050 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e18300-bf19-4a54-a98a-89c040dc2c80 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:10:25.926139 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e18300-bf19-4a54-a98a-89c040dc2c80 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:10:25.936482 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 81/162] 10.222.0.15 () {56 vars in 1111 bytes} [Thu Sep 15 12:10:25 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:10:25.937782 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e18300-bf19-4a54-a98a-89c040dc2c80 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:10:25.940782 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:10:26.030769 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d521c9fa-44fc-48b0-b99f-697c51dcc878 None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:10:26.055137 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e18300-bf19-4a54-a98a-89c040dc2c80 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:10:26.090757 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1700d65b-251d-4ace-af4f-cd9bd3217a35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:10:26.092179 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1700d65b-251d-4ace-af4f-cd9bd3217a35 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:26.092681 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1700d65b-251d-4ace-af4f-cd9bd3217a35 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:26.093199 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1700d65b-251d-4ace-af4f-cd9bd3217a35 None admin] PATH_INFO: `/v3/roles` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:26.094346 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1700d65b-251d-4ace-af4f-cd9bd3217a35 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:10:26.094861 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1700d65b-251d-4ace-af4f-cd9bd3217a35 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:10:26.096564 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1700d65b-251d-4ace-af4f-cd9bd3217a35 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:10:26.107382 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 82/163] 10.222.0.15 () {56 vars in 1099 bytes} [Thu Sep 15 12:10:25 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:10:26.109281 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1700d65b-251d-4ace-af4f-cd9bd3217a35 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:10:26.119245 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e311ee5a-cf4d-4187-a6bf-24555bada06a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:10:26.119909 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e311ee5a-cf4d-4187-a6bf-24555bada06a None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:26.120185 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e311ee5a-cf4d-4187-a6bf-24555bada06a None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:26.120460 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e311ee5a-cf4d-4187-a6bf-24555bada06a None admin] PATH_INFO: `/v3/users/5b2339bacc154a4b8cbf92a176a2a583` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:26.121685 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e311ee5a-cf4d-4187-a6bf-24555bada06a None admin] RBAC: Authorizing `identity:get_user(user_id=5b2339bacc154a4b8cbf92a176a2a583)` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:10:26.123197 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e311ee5a-cf4d-4187-a6bf-24555bada06a None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:10:26.125048 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 82/164] 10.222.0.15 () {56 vars in 1177 bytes} [Thu Sep 15 12:10:25 2022] GET /identity/v3/users/5b2339bacc154a4b8cbf92a176a2a583 => generated 289 bytes in 332 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:10:26.130014 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e311ee5a-cf4d-4187-a6bf-24555bada06a None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:10:26.189139 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:10:26.240686 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1700d65b-251d-4ace-af4f-cd9bd3217a35 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:10:26.276349 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa146086-d6bc-464f-b537-3b75bf632cff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:10:26.277189 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa146086-d6bc-464f-b537-3b75bf632cff None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:26.277683 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa146086-d6bc-464f-b537-3b75bf632cff None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:26.278051 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa146086-d6bc-464f-b537-3b75bf632cff None admin] PATH_INFO: `/v3/users/6147f6de110b41cd99def2cbeb937ad1` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:26.279299 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa146086-d6bc-464f-b537-3b75bf632cff None admin] RBAC: Authorizing `identity:get_user(user_id=6147f6de110b41cd99def2cbeb937ad1)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:10:26.280856 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa146086-d6bc-464f-b537-3b75bf632cff None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:10:26.282803 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 83/165] 10.222.0.15 () {56 vars in 1177 bytes} [Thu Sep 15 12:10:26 2022] GET /identity/v3/users/6147f6de110b41cd99def2cbeb937ad1 => generated 275 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:10:26.283686 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa146086-d6bc-464f-b537-3b75bf632cff None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:10:26.389530 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e311ee5a-cf4d-4187-a6bf-24555bada06a None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:10:26.393353 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa146086-d6bc-464f-b537-3b75bf632cff None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:10:26.410718 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:10:26.429118 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:10:26.438181 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e19964-6888-4469-be57-e6955355eeae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:10:26.439107 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e19964-6888-4469-be57-e6955355eeae None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:26.439586 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e19964-6888-4469-be57-e6955355eeae None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:26.441333 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e19964-6888-4469-be57-e6955355eeae None admin] PATH_INFO: `/v3/projects/10a21ffb0b8d4800a805c76f171c40c1` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:26.444393 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e19964-6888-4469-be57-e6955355eeae None admin] RBAC: Authorizing `identity:get_project(project_id=10a21ffb0b8d4800a805c76f171c40c1)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:10:26.444393 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e19964-6888-4469-be57-e6955355eeae None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:10:26.451414 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 84/166] 10.222.0.15 () {56 vars in 1186 bytes} [Thu Sep 15 12:10:26 2022] GET /identity/v3/projects/10a21ffb0b8d4800a805c76f171c40c1 => generated 298 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:10:26.452901 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e19964-6888-4469-be57-e6955355eeae None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:10:26.470504 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62bc84db-9831-4eab-9edb-163cb242ba13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:10:26.471786 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bc84db-9831-4eab-9edb-163cb242ba13 None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:26.472243 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bc84db-9831-4eab-9edb-163cb242ba13 None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:26.472716 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bc84db-9831-4eab-9edb-163cb242ba13 None admin] PATH_INFO: `/v3/users/c66eff9fe7124011868989f16846a501` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:26.475443 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62bc84db-9831-4eab-9edb-163cb242ba13 None admin] RBAC: Authorizing `identity:get_user(user_id=c66eff9fe7124011868989f16846a501)` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:10:26.478496 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62bc84db-9831-4eab-9edb-163cb242ba13 None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:10:26.482877 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 83/167] 10.222.0.15 () {56 vars in 1177 bytes} [Thu Sep 15 12:10:26 2022] GET /identity/v3/users/c66eff9fe7124011868989f16846a501 => generated 247 bytes in 353 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:10:26.484522 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62bc84db-9831-4eab-9edb-163cb242ba13 None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:10:26.649483 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e19964-6888-4469-be57-e6955355eeae None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:10:26.690483 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741639ef-4960-4862-bf7d-aacff7116e25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:10:26.691456 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741639ef-4960-4862-bf7d-aacff7116e25 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:26.691982 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741639ef-4960-4862-bf7d-aacff7116e25 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:26.692449 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741639ef-4960-4862-bf7d-aacff7116e25 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:26.704169 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-741639ef-4960-4862-bf7d-aacff7116e25 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 12:10:26.710647 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application [None req-741639ef-4960-4862-bf7d-aacff7116e25 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 15 12:10:26.710647 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:10:26.710647 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:10:26.710647 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:10:26.710647 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:10:26.710647 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:10:26.710647 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:10:26.710647 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:10:26.710647 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:10:26.710647 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:10:26.710647 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:10:26.710647 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:10:26.710647 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 12:10:26.710647 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 12:10:26.710647 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 12:10:26.710647 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 12:10:26.711599 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 12:10:26.711599 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 12:10:26.711599 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 12:10:26.711599 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 12:10:26.711599 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:10:26.711599 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:10:26.711599 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:10:26.711599 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:10:26.711599 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:10:26.711599 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:10:26.711599 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:10:26.711599 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application with Lock( Sep 15 12:10:26.711599 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:10:26.711599 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:10:26.711599 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:10:26.711599 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:10:26.712378 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:10:26.712378 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:10:26.712378 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:10:26.712378 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:10:26.712378 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 12:10:26.712378 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 12:10:26.712378 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 12:10:26.712378 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 12:10:26.712378 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 12:10:26.712378 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 12:10:26.712378 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 15 12:10:26.712378 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 15 12:10:26.714969 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 85/168] 10.222.0.15 () {56 vars in 1099 bytes} [Thu Sep 15 12:10:26 2022] GET /identity/v3/roles/member => generated 84 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:10:26.715900 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741639ef-4960-4862-bf7d-aacff7116e25 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:10:26.717956 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62bc84db-9831-4eab-9edb-163cb242ba13 None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:10:26.801227 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:10:26.829520 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26b2d6d-a5ec-4e74-b0c5-60225bc6a5db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:10:26.830160 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26b2d6d-a5ec-4e74-b0c5-60225bc6a5db None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:26.830275 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741639ef-4960-4862-bf7d-aacff7116e25 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:10:26.830469 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26b2d6d-a5ec-4e74-b0c5-60225bc6a5db None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:26.830708 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26b2d6d-a5ec-4e74-b0c5-60225bc6a5db None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:26.867838 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a7eb38-dc20-4b8b-b6ba-75d31851de3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:10:26.868895 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a7eb38-dc20-4b8b-b6ba-75d31851de3f None admin] REQUEST_METHOD: `PUT` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:26.869379 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a7eb38-dc20-4b8b-b6ba-75d31851de3f None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:26.869829 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a7eb38-dc20-4b8b-b6ba-75d31851de3f None admin] PATH_INFO: `/v3/projects/10a21ffb0b8d4800a805c76f171c40c1/users/5b2339bacc154a4b8cbf92a176a2a583/roles/1807da0df67b49739d033f8c0e7456f0` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:26.872142 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a7eb38-dc20-4b8b-b6ba-75d31851de3f None admin] RBAC: Authorizing `identity:create_grant(project_id=10a21ffb0b8d4800a805c76f171c40c1, user_id=5b2339bacc154a4b8cbf92a176a2a583, role_id=1807da0df67b49739d033f8c0e7456f0)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:10:26.873857 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a7eb38-dc20-4b8b-b6ba-75d31851de3f None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:10:26.903804 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 86/169] 10.222.0.15 () {58 vars in 1439 bytes} [Thu Sep 15 12:10:26 2022] PUT /identity/v3/projects/10a21ffb0b8d4800a805c76f171c40c1/users/5b2339bacc154a4b8cbf92a176a2a583/roles/1807da0df67b49739d033f8c0e7456f0 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:10:26.905132 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a7eb38-dc20-4b8b-b6ba-75d31851de3f None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:10:26.917540 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:10:26.940609 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c26b2d6d-a5ec-4e74-b0c5-60225bc6a5db None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:10:26.942282 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c26b2d6d-a5ec-4e74-b0c5-60225bc6a5db None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:10:27.106323 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 84/170] 10.222.0.15 () {58 vars in 1282 bytes} [Thu Sep 15 12:10:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 622 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 15 12:10:27.107230 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26b2d6d-a5ec-4e74-b0c5-60225bc6a5db None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:10:27.161988 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a7eb38-dc20-4b8b-b6ba-75d31851de3f None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:10:27.184398 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:10:27.207701 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:10:27.211057 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307aeb0f-2a65-4629-84b0-8fce5568598f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:10:27.211885 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307aeb0f-2a65-4629-84b0-8fce5568598f None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:27.212317 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307aeb0f-2a65-4629-84b0-8fce5568598f None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:27.212726 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307aeb0f-2a65-4629-84b0-8fce5568598f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:27.253309 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26b2d6d-a5ec-4e74-b0c5-60225bc6a5db None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:10:27.298820 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307aeb0f-2a65-4629-84b0-8fce5568598f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:10:27.300534 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307aeb0f-2a65-4629-84b0-8fce5568598f None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:10:27.339040 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e24c7a9-bb60-4c5a-b42d-7ea923928e34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:10:27.339913 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e24c7a9-bb60-4c5a-b42d-7ea923928e34 None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:27.340341 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e24c7a9-bb60-4c5a-b42d-7ea923928e34 None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:27.341037 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e24c7a9-bb60-4c5a-b42d-7ea923928e34 None admin] PATH_INFO: `/v3/roles` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:27.342879 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e24c7a9-bb60-4c5a-b42d-7ea923928e34 None admin] RBAC: Adding query filter params (name=member) {{(pid=81449) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:10:27.343354 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e24c7a9-bb60-4c5a-b42d-7ea923928e34 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:10:27.344996 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e24c7a9-bb60-4c5a-b42d-7ea923928e34 None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:10:27.358704 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 85/171] 10.222.0.15 () {56 vars in 1101 bytes} [Thu Sep 15 12:10:27 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:10:27.360171 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5250571-d266-4867-923f-8457f53bd230 None None] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:27.364777 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5250571-d266-4867-923f-8457f53bd230 None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:27.367139 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5250571-d266-4867-923f-8457f53bd230 None None] PATH_INFO: `/` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:27.368614 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 86/172] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 12:10:27 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:10:27.375209 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5250571-d266-4867-923f-8457f53bd230 None None] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:10:27.572873 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 87/173] 10.222.0.15 () {58 vars in 1282 bytes} [Thu Sep 15 12:10:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 668 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 15 12:10:27.574567 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307aeb0f-2a65-4629-84b0-8fce5568598f None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:10:27.591171 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5250571-d266-4867-923f-8457f53bd230 None None] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:10:27.636030 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:10:27.642817 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170daf8b-d1ea-41e1-bfcc-0b372be0a860 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:10:27.643837 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170daf8b-d1ea-41e1-bfcc-0b372be0a860 None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:27.644353 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170daf8b-d1ea-41e1-bfcc-0b372be0a860 None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:27.646905 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170daf8b-d1ea-41e1-bfcc-0b372be0a860 None admin] PATH_INFO: `/v3/projects/10a21ffb0b8d4800a805c76f171c40c1` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:27.648403 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170daf8b-d1ea-41e1-bfcc-0b372be0a860 None admin] RBAC: Authorizing `identity:get_project(project_id=10a21ffb0b8d4800a805c76f171c40c1)` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:10:27.651767 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170daf8b-d1ea-41e1-bfcc-0b372be0a860 None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:10:27.662845 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 87/174] 10.222.0.15 () {56 vars in 1186 bytes} [Thu Sep 15 12:10:27 2022] GET /identity/v3/projects/10a21ffb0b8d4800a805c76f171c40c1 => generated 298 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:10:27.664891 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048ebb81-9a2d-48b2-8ce0-9750d8a0c102 None None] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:27.669414 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048ebb81-9a2d-48b2-8ce0-9750d8a0c102 None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:27.670633 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048ebb81-9a2d-48b2-8ce0-9750d8a0c102 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:27.705612 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.auth.core [None req-048ebb81-9a2d-48b2-8ce0-9750d8a0c102 None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81449) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:10:27.707762 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-048ebb81-9a2d-48b2-8ce0-9750d8a0c102 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:10:27.712126 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:10:27.771166 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307aeb0f-2a65-4629-84b0-8fce5568598f None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:10:27.816920 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b97f8d-c8b2-4209-9f19-a810f6f4ede7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:10:27.818097 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b97f8d-c8b2-4209-9f19-a810f6f4ede7 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:27.818594 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b97f8d-c8b2-4209-9f19-a810f6f4ede7 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:27.819090 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b97f8d-c8b2-4209-9f19-a810f6f4ede7 None admin] PATH_INFO: `/v3/users/6147f6de110b41cd99def2cbeb937ad1` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:27.820725 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b97f8d-c8b2-4209-9f19-a810f6f4ede7 None admin] RBAC: Authorizing `identity:get_user(user_id=6147f6de110b41cd99def2cbeb937ad1)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:10:27.823261 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b97f8d-c8b2-4209-9f19-a810f6f4ede7 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:10:27.825544 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 88/175] 10.222.0.15 () {56 vars in 1177 bytes} [Thu Sep 15 12:10:27 2022] GET /identity/v3/users/6147f6de110b41cd99def2cbeb937ad1 => generated 275 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:10:27.826603 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b97f8d-c8b2-4209-9f19-a810f6f4ede7 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:10:27.878234 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 88/176] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 12:10:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 12:10:27.881529 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048ebb81-9a2d-48b2-8ce0-9750d8a0c102 None None] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:10:27.949934 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:10:27.965810 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b97f8d-c8b2-4209-9f19-a810f6f4ede7 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:10:28.001427 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4c2fb7-8efb-434a-8014-d9d73dc0bc8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:10:28.002073 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4c2fb7-8efb-434a-8014-d9d73dc0bc8c None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:28.002348 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4c2fb7-8efb-434a-8014-d9d73dc0bc8c None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:28.002617 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4c2fb7-8efb-434a-8014-d9d73dc0bc8c None admin] PATH_INFO: `/v3/projects/10a21ffb0b8d4800a805c76f171c40c1` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:28.003752 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4c2fb7-8efb-434a-8014-d9d73dc0bc8c None admin] RBAC: Authorizing `identity:get_project(project_id=10a21ffb0b8d4800a805c76f171c40c1)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:10:28.005222 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4c2fb7-8efb-434a-8014-d9d73dc0bc8c None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:10:28.007017 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 89/177] 10.222.0.15 () {56 vars in 1186 bytes} [Thu Sep 15 12:10:27 2022] GET /identity/v3/projects/10a21ffb0b8d4800a805c76f171c40c1 => generated 298 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:10:28.007627 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4c2fb7-8efb-434a-8014-d9d73dc0bc8c None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:10:28.020609 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048ebb81-9a2d-48b2-8ce0-9750d8a0c102 None None] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:10:28.061714 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e881c5-5903-4460-8794-7ce2d331fedd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:10:28.062333 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e881c5-5903-4460-8794-7ce2d331fedd None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:28.062614 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e881c5-5903-4460-8794-7ce2d331fedd None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:28.063388 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e881c5-5903-4460-8794-7ce2d331fedd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:28.064137 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e881c5-5903-4460-8794-7ce2d331fedd None admin] RBAC: Adding query filter params (role.id=1081b93ddfd547649ba8582f7a6e719e, scope.project.id=10a21ffb0b8d4800a805c76f171c40c1, user.id=6147f6de110b41cd99def2cbeb937ad1) {{(pid=81449) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:10:28.064925 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e881c5-5903-4460-8794-7ce2d331fedd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:10:28.066709 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e881c5-5903-4460-8794-7ce2d331fedd None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:10:28.074522 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 89/178] 10.222.0.15 () {56 vars in 1374 bytes} [Thu Sep 15 12:10:27 2022] GET /identity/v3/role_assignments?user.id=6147f6de110b41cd99def2cbeb937ad1&scope.project.id=10a21ffb0b8d4800a805c76f171c40c1&role.id=1081b93ddfd547649ba8582f7a6e719e => generated 608 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:10:28.076177 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293c8fb0-12cd-4501-a5c7-f6379a7e86a9 None None] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:28.076458 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293c8fb0-12cd-4501-a5c7-f6379a7e86a9 None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:28.076731 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293c8fb0-12cd-4501-a5c7-f6379a7e86a9 None None] PATH_INFO: `/` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:28.078200 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 90/179] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 12:10:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:10:28.079262 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2812531b-bdaf-4cdf-bd32-cc392b817437 None None] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:28.084027 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2812531b-bdaf-4cdf-bd32-cc392b817437 None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:28.084305 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2812531b-bdaf-4cdf-bd32-cc392b817437 None None] PATH_INFO: `/` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:28.086329 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 91/180] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 12:10:28 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:10:28.087100 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2812531b-bdaf-4cdf-bd32-cc392b817437 None None] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:10:28.090963 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:10:28.176832 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4c2fb7-8efb-434a-8014-d9d73dc0bc8c None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:10:28.227434 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add377b9-50b2-4178-b754-abad5b2df8e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:10:28.228352 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add377b9-50b2-4178-b754-abad5b2df8e7 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:28.228634 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add377b9-50b2-4178-b754-abad5b2df8e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:28.228939 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add377b9-50b2-4178-b754-abad5b2df8e7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:28.258345 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2812531b-bdaf-4cdf-bd32-cc392b817437 None None] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:10:28.261662 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:10:28.314994 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1be36d7-2b82-4978-8869-17427c219d19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:10:28.315607 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1be36d7-2b82-4978-8869-17427c219d19 None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:28.315888 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1be36d7-2b82-4978-8869-17427c219d19 None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:28.316149 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1be36d7-2b82-4978-8869-17427c219d19 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:28.316897 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1be36d7-2b82-4978-8869-17427c219d19 None admin] RBAC: Adding query filter params (role.id=9c769b5c82884766b2c510b1f0f1364c, scope.project.id=10a21ffb0b8d4800a805c76f171c40c1, user.id=c66eff9fe7124011868989f16846a501) {{(pid=81449) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:10:28.317201 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1be36d7-2b82-4978-8869-17427c219d19 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:10:28.318631 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1be36d7-2b82-4978-8869-17427c219d19 None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:10:28.324292 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add377b9-50b2-4178-b754-abad5b2df8e7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:10:28.325813 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 92/181] 10.222.0.15 () {56 vars in 1374 bytes} [Thu Sep 15 12:10:28 2022] GET /identity/v3/role_assignments?user.id=c66eff9fe7124011868989f16846a501&scope.project.id=10a21ffb0b8d4800a805c76f171c40c1&role.id=9c769b5c82884766b2c510b1f0f1364c => generated 608 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:10:28.325936 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add377b9-50b2-4178-b754-abad5b2df8e7 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:10:28.326938 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27521ed1-20b9-471e-8abc-b482063477da None None] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:28.327225 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27521ed1-20b9-471e-8abc-b482063477da None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:28.327536 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27521ed1-20b9-471e-8abc-b482063477da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:28.353494 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.auth.core [None req-27521ed1-20b9-471e-8abc-b482063477da None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81449) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:10:28.354967 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-27521ed1-20b9-471e-8abc-b482063477da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:10:28.430988 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:10:28.441981 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 93/182] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 12:10:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 12:10:28.443799 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e894f4c-fe4e-4b79-bde5-c0d64aa396f8 None None] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:28.444312 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e894f4c-fe4e-4b79-bde5-c0d64aa396f8 None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:28.444840 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e894f4c-fe4e-4b79-bde5-c0d64aa396f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:28.465475 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.auth.core [None req-7e894f4c-fe4e-4b79-bde5-c0d64aa396f8 None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81449) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:10:28.467183 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-7e894f4c-fe4e-4b79-bde5-c0d64aa396f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:10:28.474449 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 90/183] 10.222.0.15 () {58 vars in 1282 bytes} [Thu Sep 15 12:10:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 467 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 15 12:10:28.481044 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add377b9-50b2-4178-b754-abad5b2df8e7 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:10:28.546867 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 94/184] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 12:10:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 12:10:28.548101 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e894f4c-fe4e-4b79-bde5-c0d64aa396f8 None None] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:10:28.571959 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:10:28.658975 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add377b9-50b2-4178-b754-abad5b2df8e7 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:10:28.678202 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e894f4c-fe4e-4b79-bde5-c0d64aa396f8 None None] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:10:28.719731 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:10:28.732115 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4c14f9-46e0-4ad6-83de-79fe6dc94229 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:10:28.732782 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4c14f9-46e0-4ad6-83de-79fe6dc94229 None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:28.733059 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4c14f9-46e0-4ad6-83de-79fe6dc94229 None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:28.733332 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4c14f9-46e0-4ad6-83de-79fe6dc94229 None admin] PATH_INFO: `/v3/projects/53dad4f5e490445fa10339bdf7c51660` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:28.734462 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4c14f9-46e0-4ad6-83de-79fe6dc94229 None admin] RBAC: Authorizing `identity:get_project(project_id=53dad4f5e490445fa10339bdf7c51660)` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:10:28.736628 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4c14f9-46e0-4ad6-83de-79fe6dc94229 None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:10:28.738399 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 95/185] 10.222.0.15 () {56 vars in 1186 bytes} [Thu Sep 15 12:10:28 2022] GET /identity/v3/projects/53dad4f5e490445fa10339bdf7c51660 => generated 312 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:10:28.744711 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95565d9f-33b1-4d1c-b760-629fc66fa448 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:10:28.745078 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4c14f9-46e0-4ad6-83de-79fe6dc94229 None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:10:28.745389 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95565d9f-33b1-4d1c-b760-629fc66fa448 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:28.745669 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95565d9f-33b1-4d1c-b760-629fc66fa448 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:28.745936 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95565d9f-33b1-4d1c-b760-629fc66fa448 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:28.763323 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95565d9f-33b1-4d1c-b760-629fc66fa448 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 12:10:28.768270 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application [None req-95565d9f-33b1-4d1c-b760-629fc66fa448 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 15 12:10:28.768270 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:10:28.768270 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:10:28.768270 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:10:28.768270 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:10:28.768270 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:10:28.768270 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:10:28.768270 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:10:28.768270 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:10:28.768270 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:10:28.768270 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:10:28.768270 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:10:28.768270 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 12:10:28.768270 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 12:10:28.768270 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 12:10:28.768270 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 12:10:28.769202 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 12:10:28.769202 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 12:10:28.769202 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 12:10:28.769202 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 12:10:28.769202 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:10:28.769202 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:10:28.769202 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:10:28.769202 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:10:28.769202 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:10:28.769202 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:10:28.769202 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:10:28.769202 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application with Lock( Sep 15 12:10:28.769202 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:10:28.769202 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:10:28.769202 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:10:28.769202 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:10:28.770002 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:10:28.770002 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:10:28.770002 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:10:28.770002 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:10:28.770002 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 12:10:28.770002 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 12:10:28.770002 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 12:10:28.770002 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 12:10:28.770002 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 12:10:28.770002 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 12:10:28.770002 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 15 12:10:28.770002 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 15 12:10:28.770002 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 91/186] 10.222.0.15 () {56 vars in 1096 bytes} [Thu Sep 15 12:10:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 294 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:10:28.782722 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95565d9f-33b1-4d1c-b760-629fc66fa448 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:10:28.914384 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4c14f9-46e0-4ad6-83de-79fe6dc94229 None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:10:28.932780 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95565d9f-33b1-4d1c-b760-629fc66fa448 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:10:28.949114 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:10:28.973775 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a43f085-01ae-4728-9980-c0fd887df243 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:10:28.974398 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a43f085-01ae-4728-9980-c0fd887df243 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:28.974677 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a43f085-01ae-4728-9980-c0fd887df243 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:28.974947 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a43f085-01ae-4728-9980-c0fd887df243 None admin] PATH_INFO: `/v3/roles` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:28.976368 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a43f085-01ae-4728-9980-c0fd887df243 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:10:28.976669 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a43f085-01ae-4728-9980-c0fd887df243 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:10:28.978114 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a43f085-01ae-4728-9980-c0fd887df243 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:10:28.983458 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edda0b2c-81dc-44fb-80ae-ed90974359da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:10:28.984068 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edda0b2c-81dc-44fb-80ae-ed90974359da None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:28.984341 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edda0b2c-81dc-44fb-80ae-ed90974359da None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:28.984607 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edda0b2c-81dc-44fb-80ae-ed90974359da None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:28.985351 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edda0b2c-81dc-44fb-80ae-ed90974359da None admin] RBAC: Adding query filter params (role.id=0679d6628a25438d90dc4bdc0401eda2, scope.project.id=53dad4f5e490445fa10339bdf7c51660, user.id=6147f6de110b41cd99def2cbeb937ad1) {{(pid=81449) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:10:28.985652 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edda0b2c-81dc-44fb-80ae-ed90974359da None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:10:28.987060 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edda0b2c-81dc-44fb-80ae-ed90974359da None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:10:28.987856 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 92/187] 10.222.0.15 () {56 vars in 1099 bytes} [Thu Sep 15 12:10:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 12:10:28.989540 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a43f085-01ae-4728-9980-c0fd887df243 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:10:28.997144 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 96/188] 10.222.0.15 () {56 vars in 1374 bytes} [Thu Sep 15 12:10:28 2022] GET /identity/v3/role_assignments?user.id=6147f6de110b41cd99def2cbeb937ad1&scope.project.id=53dad4f5e490445fa10339bdf7c51660&role.id=0679d6628a25438d90dc4bdc0401eda2 => generated 608 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 12:10:28.998180 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edda0b2c-81dc-44fb-80ae-ed90974359da None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:10:29.077263 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:10:29.116918 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a43f085-01ae-4728-9980-c0fd887df243 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:10:29.142770 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edda0b2c-81dc-44fb-80ae-ed90974359da None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:10:29.159025 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663622ef-e606-4861-a5cf-943b769ef4f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:10:29.159830 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663622ef-e606-4861-a5cf-943b769ef4f6 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:29.160195 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663622ef-e606-4861-a5cf-943b769ef4f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:29.160537 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663622ef-e606-4861-a5cf-943b769ef4f6 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:29.174883 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663622ef-e606-4861-a5cf-943b769ef4f6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 12:10:29.181819 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application [None req-663622ef-e606-4861-a5cf-943b769ef4f6 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 15 12:10:29.181819 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:10:29.181819 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:10:29.181819 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:10:29.181819 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:10:29.181819 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:10:29.181819 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:10:29.181819 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:10:29.181819 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:10:29.181819 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:10:29.181819 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:10:29.181819 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:10:29.181819 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 12:10:29.181819 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 12:10:29.181819 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 12:10:29.181819 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 12:10:29.182740 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 12:10:29.182740 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 12:10:29.182740 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 12:10:29.182740 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 12:10:29.182740 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:10:29.182740 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:10:29.182740 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:10:29.182740 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:10:29.182740 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:10:29.182740 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:10:29.182740 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:10:29.182740 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application with Lock( Sep 15 12:10:29.182740 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:10:29.182740 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:10:29.182740 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:10:29.182740 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:10:29.184184 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:10:29.184184 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:10:29.184184 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:10:29.184184 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:10:29.184184 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 12:10:29.184184 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 12:10:29.184184 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 12:10:29.184184 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 12:10:29.184184 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 12:10:29.184184 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 12:10:29.184184 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 15 12:10:29.184184 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 15 12:10:29.184184 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 93/189] 10.222.0.15 () {56 vars in 1114 bytes} [Thu Sep 15 12:10:28 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:10:29.184184 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663622ef-e606-4861-a5cf-943b769ef4f6 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:10:29.192743 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b5da5f-dc9e-4ca7-80e7-6a8eae073aee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:10:29.193443 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b5da5f-dc9e-4ca7-80e7-6a8eae073aee None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:29.193724 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b5da5f-dc9e-4ca7-80e7-6a8eae073aee None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:29.193995 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b5da5f-dc9e-4ca7-80e7-6a8eae073aee None admin] PATH_INFO: `/v3/domains/default` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:29.195590 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b5da5f-dc9e-4ca7-80e7-6a8eae073aee None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:10:29.197099 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b5da5f-dc9e-4ca7-80e7-6a8eae073aee None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:10:29.199094 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 97/190] 10.222.0.15 () {56 vars in 1108 bytes} [Thu Sep 15 12:10:28 2022] GET /identity/v3/domains/default => generated 199 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:10:29.200344 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b5da5f-dc9e-4ca7-80e7-6a8eae073aee None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:10:29.240498 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:10:29.314104 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663622ef-e606-4861-a5cf-943b769ef4f6 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:10:29.318218 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b5da5f-dc9e-4ca7-80e7-6a8eae073aee None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:10:29.346338 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:10:29.355122 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a558fb6d-3e97-496e-b05a-56db419948df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:10:29.355795 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a558fb6d-3e97-496e-b05a-56db419948df None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:29.356072 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a558fb6d-3e97-496e-b05a-56db419948df None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:29.356343 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a558fb6d-3e97-496e-b05a-56db419948df None admin] PATH_INFO: `/v3/users/42017b63f1c045cdad77de4fceaa6cb8` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:29.356986 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e2688f-5ac2-402a-8118-f1ba624dd668 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:10:29.357621 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e2688f-5ac2-402a-8118-f1ba624dd668 None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:29.357906 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e2688f-5ac2-402a-8118-f1ba624dd668 None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:29.358182 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e2688f-5ac2-402a-8118-f1ba624dd668 None admin] PATH_INFO: `/v3/roles` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:29.358873 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e2688f-5ac2-402a-8118-f1ba624dd668 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81449) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:10:29.359179 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e2688f-5ac2-402a-8118-f1ba624dd668 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:10:29.360605 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e2688f-5ac2-402a-8118-f1ba624dd668 None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:10:29.369801 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 98/191] 10.222.0.15 () {56 vars in 1111 bytes} [Thu Sep 15 12:10:29 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:10:29.370438 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e2688f-5ac2-402a-8118-f1ba624dd668 None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:10:29.374156 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a558fb6d-3e97-496e-b05a-56db419948df None admin] RBAC: Authorizing `identity:get_user(user_id=42017b63f1c045cdad77de4fceaa6cb8)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:10:29.376073 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a558fb6d-3e97-496e-b05a-56db419948df None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:10:29.377274 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 94/192] 10.222.0.15 () {56 vars in 1177 bytes} [Thu Sep 15 12:10:29 2022] GET /identity/v3/users/42017b63f1c045cdad77de4fceaa6cb8 => generated 283 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:10:29.378034 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a558fb6d-3e97-496e-b05a-56db419948df None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:10:29.476989 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e2688f-5ac2-402a-8118-f1ba624dd668 None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:10:29.478733 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:10:29.485550 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a558fb6d-3e97-496e-b05a-56db419948df None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:10:29.510584 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e1c5d1-e088-42bc-b931-654e41d9e9c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:10:29.511201 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e1c5d1-e088-42bc-b931-654e41d9e9c7 None admin] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:29.511479 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e1c5d1-e088-42bc-b931-654e41d9e9c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:29.511748 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e1c5d1-e088-42bc-b931-654e41d9e9c7 None admin] PATH_INFO: `/v3/users` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:29.512392 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e1c5d1-e088-42bc-b931-654e41d9e9c7 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:10:29.513911 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e1c5d1-e088-42bc-b931-654e41d9e9c7 None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:10:29.520215 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167bf6bb-1b34-467e-8a26-b4f6c56f34c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:10:29.520875 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167bf6bb-1b34-467e-8a26-b4f6c56f34c2 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:29.521158 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167bf6bb-1b34-467e-8a26-b4f6c56f34c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:29.521431 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167bf6bb-1b34-467e-8a26-b4f6c56f34c2 None admin] PATH_INFO: `/v3/users/42017b63f1c045cdad77de4fceaa6cb8` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:29.522571 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167bf6bb-1b34-467e-8a26-b4f6c56f34c2 None admin] RBAC: Authorizing `identity:get_user(user_id=42017b63f1c045cdad77de4fceaa6cb8)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:10:29.524087 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167bf6bb-1b34-467e-8a26-b4f6c56f34c2 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:10:29.525918 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 95/193] 10.222.0.15 () {56 vars in 1177 bytes} [Thu Sep 15 12:10:29 2022] GET /identity/v3/users/42017b63f1c045cdad77de4fceaa6cb8 => generated 283 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:10:29.526734 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167bf6bb-1b34-467e-8a26-b4f6c56f34c2 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:10:29.546338 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 99/194] 10.222.0.15 () {60 vars in 1132 bytes} [Thu Sep 15 12:10:29 2022] POST /identity/v3/users => generated 297 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 12:10:29.546338 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bc30f5-9deb-47d0-80c5-10c476b212ef None None] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:29.546338 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bc30f5-9deb-47d0-80c5-10c476b212ef None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:29.546338 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bc30f5-9deb-47d0-80c5-10c476b212ef None None] PATH_INFO: `/` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:29.546338 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 100/195] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 12:10:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:10:29.546338 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47bc30f5-9deb-47d0-80c5-10c476b212ef None None] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:10:29.597446 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:10:29.645186 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167bf6bb-1b34-467e-8a26-b4f6c56f34c2 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:10:29.665189 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47bc30f5-9deb-47d0-80c5-10c476b212ef None None] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:10:29.682485 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23bfac01-95cb-444d-a9e4-6ae793cf104e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:10:29.683387 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bfac01-95cb-444d-a9e4-6ae793cf104e None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:29.683387 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bfac01-95cb-444d-a9e4-6ae793cf104e None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:29.683636 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bfac01-95cb-444d-a9e4-6ae793cf104e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:29.713339 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f402605-ac8b-4766-b13b-c499e842c7b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:10:29.713884 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f402605-ac8b-4766-b13b-c499e842c7b1 None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:29.714123 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f402605-ac8b-4766-b13b-c499e842c7b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:29.714385 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f402605-ac8b-4766-b13b-c499e842c7b1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:29.723298 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:10:29.765911 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23bfac01-95cb-444d-a9e4-6ae793cf104e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:10:29.767281 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23bfac01-95cb-444d-a9e4-6ae793cf104e None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:10:29.799165 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f402605-ac8b-4766-b13b-c499e842c7b1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:10:29.800539 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f402605-ac8b-4766-b13b-c499e842c7b1 None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:10:29.854989 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:10:29.895363 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 96/196] 10.222.0.15 () {58 vars in 1282 bytes} [Thu Sep 15 12:10:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 15 12:10:29.901861 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d54065-b337-4b62-b776-cbd6229012aa None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:29.902155 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d54065-b337-4b62-b776-cbd6229012aa None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:29.902430 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d54065-b337-4b62-b776-cbd6229012aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:29.921222 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-b8d54065-b337-4b62-b776-cbd6229012aa None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:10:29.921747 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 101/197] 10.222.0.15 () {58 vars in 1282 bytes} [Thu Sep 15 12:10:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 15 12:10:29.922370 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f402605-ac8b-4766-b13b-c499e842c7b1 None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:10:29.922490 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-b8d54065-b337-4b62-b776-cbd6229012aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:10:29.998094 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:10:30.014201 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 97/198] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 12:10:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 12:10:30.015401 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d54065-b337-4b62-b776-cbd6229012aa None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:10:30.083654 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f402605-ac8b-4766-b13b-c499e842c7b1 None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:10:30.122882 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622e0e47-efe0-487d-822c-678992dc7718 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:10:30.123912 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622e0e47-efe0-487d-822c-678992dc7718 None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:30.124386 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622e0e47-efe0-487d-822c-678992dc7718 None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:30.124920 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622e0e47-efe0-487d-822c-678992dc7718 None admin] PATH_INFO: `/v3/projects/bd9c06e0f78e4d26902fd521a2fe013e` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:30.126601 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622e0e47-efe0-487d-822c-678992dc7718 None admin] RBAC: Authorizing `identity:get_project(project_id=bd9c06e0f78e4d26902fd521a2fe013e)` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:10:30.128520 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622e0e47-efe0-487d-822c-678992dc7718 None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:10:30.130380 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 102/199] 10.222.0.15 () {56 vars in 1186 bytes} [Thu Sep 15 12:10:29 2022] GET /identity/v3/projects/bd9c06e0f78e4d26902fd521a2fe013e => generated 302 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:10:30.136964 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622e0e47-efe0-487d-822c-678992dc7718 None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:10:30.154497 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:10:30.211093 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d54065-b337-4b62-b776-cbd6229012aa None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:10:30.247789 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbd0896-6055-419a-a467-ab9558556339 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:10:30.248797 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbd0896-6055-419a-a467-ab9558556339 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:30.249275 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbd0896-6055-419a-a467-ab9558556339 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:30.249774 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbd0896-6055-419a-a467-ab9558556339 None admin] PATH_INFO: `/v3/projects/bd9c06e0f78e4d26902fd521a2fe013e` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:30.251180 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccbd0896-6055-419a-a467-ab9558556339 None admin] RBAC: Authorizing `identity:get_project(project_id=bd9c06e0f78e4d26902fd521a2fe013e)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:10:30.252876 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccbd0896-6055-419a-a467-ab9558556339 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:10:30.254823 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 98/200] 10.222.0.15 () {56 vars in 1186 bytes} [Thu Sep 15 12:10:30 2022] GET /identity/v3/projects/bd9c06e0f78e4d26902fd521a2fe013e => generated 302 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:10:30.261244 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbd0896-6055-419a-a467-ab9558556339 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:10:30.334025 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:10:30.336876 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622e0e47-efe0-487d-822c-678992dc7718 None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:10:30.376656 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e77b6c-db00-4606-8e5a-fe6a3931fc44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:10:30.377668 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e77b6c-db00-4606-8e5a-fe6a3931fc44 None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:30.378192 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e77b6c-db00-4606-8e5a-fe6a3931fc44 None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:30.378692 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e77b6c-db00-4606-8e5a-fe6a3931fc44 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:30.379662 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e77b6c-db00-4606-8e5a-fe6a3931fc44 None admin] RBAC: Adding query filter params (role.id=9c769b5c82884766b2c510b1f0f1364c, scope.project.id=bd9c06e0f78e4d26902fd521a2fe013e, user.id=42017b63f1c045cdad77de4fceaa6cb8) {{(pid=81449) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:10:30.380178 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e77b6c-db00-4606-8e5a-fe6a3931fc44 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:10:30.382740 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e77b6c-db00-4606-8e5a-fe6a3931fc44 None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:10:30.385513 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbd0896-6055-419a-a467-ab9558556339 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:10:30.389394 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 103/201] 10.222.0.15 () {56 vars in 1374 bytes} [Thu Sep 15 12:10:30 2022] GET /identity/v3/role_assignments?user.id=42017b63f1c045cdad77de4fceaa6cb8&scope.project.id=bd9c06e0f78e4d26902fd521a2fe013e&role.id=9c769b5c82884766b2c510b1f0f1364c => generated 259 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:10:30.393854 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e77b6c-db00-4606-8e5a-fe6a3931fc44 None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:10:30.427435 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c3ecfc-b6fa-4cc5-8dfa-b8a1bcc79f66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:10:30.428065 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c3ecfc-b6fa-4cc5-8dfa-b8a1bcc79f66 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:30.428345 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c3ecfc-b6fa-4cc5-8dfa-b8a1bcc79f66 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:30.429193 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c3ecfc-b6fa-4cc5-8dfa-b8a1bcc79f66 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:30.429942 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c3ecfc-b6fa-4cc5-8dfa-b8a1bcc79f66 None admin] RBAC: Adding query filter params (role.id=1081b93ddfd547649ba8582f7a6e719e, scope.project.id=bd9c06e0f78e4d26902fd521a2fe013e, user.id=42017b63f1c045cdad77de4fceaa6cb8) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:10:30.430276 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c3ecfc-b6fa-4cc5-8dfa-b8a1bcc79f66 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:10:30.432864 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c3ecfc-b6fa-4cc5-8dfa-b8a1bcc79f66 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:10:30.439842 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 99/202] 10.222.0.15 () {56 vars in 1374 bytes} [Thu Sep 15 12:10:30 2022] GET /identity/v3/role_assignments?user.id=42017b63f1c045cdad77de4fceaa6cb8&scope.project.id=bd9c06e0f78e4d26902fd521a2fe013e&role.id=1081b93ddfd547649ba8582f7a6e719e => generated 259 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:10:30.522951 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:10:30.567215 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e77b6c-db00-4606-8e5a-fe6a3931fc44 None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:10:30.612451 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e5b1db-5e2b-4c08-b3bc-4c384b44bd6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:10:30.613515 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e5b1db-5e2b-4c08-b3bc-4c384b44bd6f None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:30.614010 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e5b1db-5e2b-4c08-b3bc-4c384b44bd6f None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:30.614483 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e5b1db-5e2b-4c08-b3bc-4c384b44bd6f None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:30.638226 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e5b1db-5e2b-4c08-b3bc-4c384b44bd6f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 12:10:30.644198 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application [None req-97e5b1db-5e2b-4c08-b3bc-4c384b44bd6f None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 15 12:10:30.644198 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:10:30.644198 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:10:30.644198 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:10:30.644198 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:10:30.644198 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:10:30.644198 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:10:30.644198 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:10:30.644198 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:10:30.644198 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:10:30.644198 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:10:30.644198 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:10:30.644198 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 12:10:30.644198 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 12:10:30.644198 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 12:10:30.644198 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 12:10:30.645174 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 12:10:30.645174 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 12:10:30.645174 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 12:10:30.645174 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 12:10:30.645174 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:10:30.645174 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:10:30.645174 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:10:30.645174 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:10:30.645174 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:10:30.645174 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:10:30.645174 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:10:30.645174 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application with Lock( Sep 15 12:10:30.645174 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:10:30.645174 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:10:30.645174 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:10:30.645174 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:10:30.646000 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:10:30.646000 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:10:30.646000 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:10:30.646000 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:10:30.646000 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 12:10:30.646000 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 12:10:30.646000 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 12:10:30.646000 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 12:10:30.646000 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 12:10:30.646000 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 12:10:30.646000 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 15 12:10:30.646000 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application Sep 15 12:10:30.648443 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 104/203] 10.222.0.15 () {56 vars in 1099 bytes} [Thu Sep 15 12:10:30 2022] GET /identity/v3/roles/reader => generated 84 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:10:30.654831 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c3ecfc-b6fa-4cc5-8dfa-b8a1bcc79f66 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:10:30.791667 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c3ecfc-b6fa-4cc5-8dfa-b8a1bcc79f66 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:10:30.822572 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:10:30.827051 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c1a8fa-1ff9-4648-bbce-e1bbf431d1a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:10:30.827675 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c1a8fa-1ff9-4648-bbce-e1bbf431d1a5 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:30.827956 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c1a8fa-1ff9-4648-bbce-e1bbf431d1a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:30.828229 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c1a8fa-1ff9-4648-bbce-e1bbf431d1a5 None admin] PATH_INFO: `/v3/roles` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:30.828918 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c1a8fa-1ff9-4648-bbce-e1bbf431d1a5 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:10:30.829222 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c1a8fa-1ff9-4648-bbce-e1bbf431d1a5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:10:30.830654 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c1a8fa-1ff9-4648-bbce-e1bbf431d1a5 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:10:30.840495 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 100/204] 10.222.0.15 () {56 vars in 1101 bytes} [Thu Sep 15 12:10:30 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:10:30.846180 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e5b1db-5e2b-4c08-b3bc-4c384b44bd6f None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:10:30.959890 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e5b1db-5e2b-4c08-b3bc-4c384b44bd6f None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:10:31.004613 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0076bf1d-821b-44f9-b2f1-6828d5461d5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:10:31.005278 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0076bf1d-821b-44f9-b2f1-6828d5461d5e None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:31.005558 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0076bf1d-821b-44f9-b2f1-6828d5461d5e None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:31.005829 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0076bf1d-821b-44f9-b2f1-6828d5461d5e None admin] PATH_INFO: `/v3/users/5b2339bacc154a4b8cbf92a176a2a583` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:31.009446 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0076bf1d-821b-44f9-b2f1-6828d5461d5e None admin] RBAC: Authorizing `identity:get_user(user_id=5b2339bacc154a4b8cbf92a176a2a583)` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:10:31.010919 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0076bf1d-821b-44f9-b2f1-6828d5461d5e None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:10:31.012600 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 105/205] 10.222.0.15 () {56 vars in 1177 bytes} [Thu Sep 15 12:10:30 2022] GET /identity/v3/users/5b2339bacc154a4b8cbf92a176a2a583 => generated 289 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:10:31.018201 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c1a8fa-1ff9-4648-bbce-e1bbf431d1a5 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:10:31.130999 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c1a8fa-1ff9-4648-bbce-e1bbf431d1a5 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:10:31.165699 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7869f7f0-b9e8-4afa-8649-90d24126b8cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:10:31.166323 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7869f7f0-b9e8-4afa-8649-90d24126b8cc None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:31.166602 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7869f7f0-b9e8-4afa-8649-90d24126b8cc None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:31.166872 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7869f7f0-b9e8-4afa-8649-90d24126b8cc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:31.230385 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:10:31.237660 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7869f7f0-b9e8-4afa-8649-90d24126b8cc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:10:31.239192 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7869f7f0-b9e8-4afa-8649-90d24126b8cc None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:10:31.339990 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 101/206] 10.222.0.15 () {58 vars in 1282 bytes} [Thu Sep 15 12:10:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 15 12:10:31.345782 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0076bf1d-821b-44f9-b2f1-6828d5461d5e None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:10:31.350717 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:10:31.469573 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0076bf1d-821b-44f9-b2f1-6828d5461d5e None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:10:31.518846 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff2bc36-266c-4412-a635-a116b2918b07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:10:31.519406 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff2bc36-266c-4412-a635-a116b2918b07 None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:31.519686 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff2bc36-266c-4412-a635-a116b2918b07 None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:31.519957 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff2bc36-266c-4412-a635-a116b2918b07 None admin] PATH_INFO: `/v3/projects/10a21ffb0b8d4800a805c76f171c40c1` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:31.521176 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff2bc36-266c-4412-a635-a116b2918b07 None admin] RBAC: Authorizing `identity:get_project(project_id=10a21ffb0b8d4800a805c76f171c40c1)` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:10:31.523304 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff2bc36-266c-4412-a635-a116b2918b07 None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:10:31.525266 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 106/207] 10.222.0.15 () {56 vars in 1186 bytes} [Thu Sep 15 12:10:31 2022] GET /identity/v3/projects/10a21ffb0b8d4800a805c76f171c40c1 => generated 298 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:10:31.531244 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7869f7f0-b9e8-4afa-8649-90d24126b8cc None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:10:31.627274 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:10:31.639858 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7869f7f0-b9e8-4afa-8649-90d24126b8cc None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:10:31.675286 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dde404c-242d-4877-b67c-5dac4721fb02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:10:31.675909 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dde404c-242d-4877-b67c-5dac4721fb02 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:31.676186 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dde404c-242d-4877-b67c-5dac4721fb02 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:31.676456 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dde404c-242d-4877-b67c-5dac4721fb02 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:31.677214 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dde404c-242d-4877-b67c-5dac4721fb02 None admin] RBAC: Adding query filter params (role.id=1807da0df67b49739d033f8c0e7456f0, scope.project.id=10a21ffb0b8d4800a805c76f171c40c1, user.id=5b2339bacc154a4b8cbf92a176a2a583) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:10:31.677521 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dde404c-242d-4877-b67c-5dac4721fb02 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:10:31.679524 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dde404c-242d-4877-b67c-5dac4721fb02 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:10:31.688713 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 102/208] 10.222.0.15 () {56 vars in 1374 bytes} [Thu Sep 15 12:10:31 2022] GET /identity/v3/role_assignments?user.id=5b2339bacc154a4b8cbf92a176a2a583&scope.project.id=10a21ffb0b8d4800a805c76f171c40c1&role.id=1807da0df67b49739d033f8c0e7456f0 => generated 608 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:10:32.214657 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f504891f-5b4c-4fe3-9328-e6fac89dc0c6 None None] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:32.215197 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f504891f-5b4c-4fe3-9328-e6fac89dc0c6 None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:32.215197 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f504891f-5b4c-4fe3-9328-e6fac89dc0c6 None None] PATH_INFO: `/` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:32.216810 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 107/209] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 12:10:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:10:32.222390 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aed71e0-65c8-4873-a073-3207126ea9aa None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:32.222785 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aed71e0-65c8-4873-a073-3207126ea9aa None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:32.223023 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aed71e0-65c8-4873-a073-3207126ea9aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:32.242857 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-1aed71e0-65c8-4873-a073-3207126ea9aa None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:10:32.243989 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-1aed71e0-65c8-4873-a073-3207126ea9aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:10:32.322351 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 103/210] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 12:10:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 12:10:32.422560 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ceaf3f-d011-4b02-9a71-2410a34473d1 None None] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:32.422905 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ceaf3f-d011-4b02-9a71-2410a34473d1 None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:32.423116 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ceaf3f-d011-4b02-9a71-2410a34473d1 None None] PATH_INFO: `/` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:32.424810 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 108/211] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 12:10:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:10:32.429661 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfa221b-26d9-4348-a8a9-9533383de4dd None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:32.429945 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfa221b-26d9-4348-a8a9-9533383de4dd None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:32.430219 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfa221b-26d9-4348-a8a9-9533383de4dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:32.448786 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-0bfa221b-26d9-4348-a8a9-9533383de4dd None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:10:32.449947 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-0bfa221b-26d9-4348-a8a9-9533383de4dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:10:32.517616 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 104/212] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 12:10:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 12:10:32.605173 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ceaf3f-d011-4b02-9a71-2410a34473d1 None None] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:10:32.682510 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:10:32.717393 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ceaf3f-d011-4b02-9a71-2410a34473d1 None None] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:10:32.753176 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a98eeac-6a97-4758-bc7a-0ef2badb4791 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:10:32.753808 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a98eeac-6a97-4758-bc7a-0ef2badb4791 None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:32.754083 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a98eeac-6a97-4758-bc7a-0ef2badb4791 None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:32.754347 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a98eeac-6a97-4758-bc7a-0ef2badb4791 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:32.765327 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a98eeac-6a97-4758-bc7a-0ef2badb4791 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 12:10:32.770700 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application [None req-6a98eeac-6a97-4758-bc7a-0ef2badb4791 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 15 12:10:32.770700 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:10:32.770700 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:10:32.770700 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:10:32.770700 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:10:32.770700 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:10:32.770700 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:10:32.770700 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:10:32.770700 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:10:32.770700 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:10:32.770700 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:10:32.770700 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:10:32.770700 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 12:10:32.770700 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 12:10:32.770700 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 12:10:32.770700 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 12:10:32.771570 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 12:10:32.771570 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 12:10:32.771570 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 12:10:32.771570 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 12:10:32.771570 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:10:32.771570 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:10:32.771570 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:10:32.771570 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:10:32.771570 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:10:32.771570 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:10:32.771570 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:10:32.771570 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application with Lock( Sep 15 12:10:32.771570 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:10:32.771570 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:10:32.771570 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:10:32.771570 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:10:32.772976 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:10:32.772976 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:10:32.772976 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:10:32.772976 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:10:32.772976 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 12:10:32.772976 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 12:10:32.772976 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 12:10:32.772976 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 12:10:32.772976 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 12:10:32.772976 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 12:10:32.772976 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 15 12:10:32.772976 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application Sep 15 12:10:32.772976 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 109/213] 10.222.0.15 () {56 vars in 1099 bytes} [Thu Sep 15 12:10:32 2022] GET /identity/v3/roles/member => generated 84 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:10:32.778070 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bfa221b-26d9-4348-a8a9-9533383de4dd None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:10:32.793587 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:10:32.797608 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a98eeac-6a97-4758-bc7a-0ef2badb4791 None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:10:32.885836 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bfa221b-26d9-4348-a8a9-9533383de4dd None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:10:32.922830 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c7280e-661a-4f94-ba52-3789aba02d76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:10:32.923450 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c7280e-661a-4f94-ba52-3789aba02d76 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:32.923729 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c7280e-661a-4f94-ba52-3789aba02d76 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:32.923999 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c7280e-661a-4f94-ba52-3789aba02d76 None admin] PATH_INFO: `/v3/roles` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:32.924663 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c7280e-661a-4f94-ba52-3789aba02d76 None admin] RBAC: Adding query filter params (name=member) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:10:32.924993 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c7280e-661a-4f94-ba52-3789aba02d76 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:10:32.926371 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c7280e-661a-4f94-ba52-3789aba02d76 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:10:32.936518 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 105/214] 10.222.0.15 () {56 vars in 1101 bytes} [Thu Sep 15 12:10:32 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:10:32.942503 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c7280e-661a-4f94-ba52-3789aba02d76 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:10:32.946128 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a98eeac-6a97-4758-bc7a-0ef2badb4791 None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:10:32.995330 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:10:33.003359 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1386d8-1a45-432e-b9e2-659e7ce798a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:10:33.004665 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1386d8-1a45-432e-b9e2-659e7ce798a1 None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:33.005136 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1386d8-1a45-432e-b9e2-659e7ce798a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:33.006051 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1386d8-1a45-432e-b9e2-659e7ce798a1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:33.007548 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1386d8-1a45-432e-b9e2-659e7ce798a1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:10:33.009982 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1386d8-1a45-432e-b9e2-659e7ce798a1 None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:10:33.012312 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 110/215] 10.222.0.15 () {56 vars in 1108 bytes} [Thu Sep 15 12:10:32 2022] GET /identity/v3/domains/default => generated 199 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:10:33.018640 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1386d8-1a45-432e-b9e2-659e7ce798a1 None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:10:33.050413 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:10:33.087515 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c7280e-661a-4f94-ba52-3789aba02d76 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:10:33.130605 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1386d8-1a45-432e-b9e2-659e7ce798a1 None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:10:33.130927 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8815398-3ab1-4f5e-9bec-886f6289fcac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:10:33.131536 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8815398-3ab1-4f5e-9bec-886f6289fcac None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:33.131856 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8815398-3ab1-4f5e-9bec-886f6289fcac None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:33.132134 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8815398-3ab1-4f5e-9bec-886f6289fcac None admin] PATH_INFO: `/v3/users/b8b4429c9fff4109b34e2ffafd7ba319` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:33.145387 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8815398-3ab1-4f5e-9bec-886f6289fcac None admin] RBAC: Authorizing `identity:get_user(user_id=b8b4429c9fff4109b34e2ffafd7ba319)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:10:33.146827 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8815398-3ab1-4f5e-9bec-886f6289fcac None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:10:33.149168 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 106/216] 10.222.0.15 () {56 vars in 1177 bytes} [Thu Sep 15 12:10:32 2022] GET /identity/v3/users/b8b4429c9fff4109b34e2ffafd7ba319 => generated 297 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 12:10:33.154554 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1545a6a-c048-4b30-8ce7-6f60783aeb42 None None] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:33.154846 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1545a6a-c048-4b30-8ce7-6f60783aeb42 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:33.155114 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1545a6a-c048-4b30-8ce7-6f60783aeb42 None None] PATH_INFO: `/` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:33.156356 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 107/217] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 12:10:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:10:33.157068 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1545a6a-c048-4b30-8ce7-6f60783aeb42 None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:10:33.169680 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3420707-905c-47df-9fa3-7fb2dbededb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:10:33.170291 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3420707-905c-47df-9fa3-7fb2dbededb6 None admin] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:33.170568 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3420707-905c-47df-9fa3-7fb2dbededb6 None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:33.170838 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3420707-905c-47df-9fa3-7fb2dbededb6 None admin] PATH_INFO: `/v3/users` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:33.171486 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3420707-905c-47df-9fa3-7fb2dbededb6 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:10:33.172945 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3420707-905c-47df-9fa3-7fb2dbededb6 None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:10:33.195572 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 111/218] 10.222.0.15 () {60 vars in 1132 bytes} [Thu Sep 15 12:10:33 2022] POST /identity/v3/users => generated 297 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 15 12:10:33.195572 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2007a9a1-aba8-4391-98d3-6e322e59c637 None None] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:33.195572 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2007a9a1-aba8-4391-98d3-6e322e59c637 None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:33.195572 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2007a9a1-aba8-4391-98d3-6e322e59c637 None None] PATH_INFO: `/` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:33.195572 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 112/219] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 12:10:33 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:10:33.195572 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7354e15b-84e7-4621-aef1-411ecb19fa29 None None] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:33.195572 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7354e15b-84e7-4621-aef1-411ecb19fa29 None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:33.202954 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7354e15b-84e7-4621-aef1-411ecb19fa29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:33.219289 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.auth.core [None req-7354e15b-84e7-4621-aef1-411ecb19fa29 None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81449) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:10:33.220507 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-7354e15b-84e7-4621-aef1-411ecb19fa29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:10:33.277923 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:10:33.288791 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1545a6a-c048-4b30-8ce7-6f60783aeb42 None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:10:33.304883 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 113/220] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 12:10:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 12:10:33.304883 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386fb385-ca8f-41ab-b361-b5ff22e6dd0a None None] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:33.304883 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386fb385-ca8f-41ab-b361-b5ff22e6dd0a None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:33.304883 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386fb385-ca8f-41ab-b361-b5ff22e6dd0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:33.324520 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.auth.core [None req-386fb385-ca8f-41ab-b361-b5ff22e6dd0a None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81449) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:10:33.326844 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-386fb385-ca8f-41ab-b361-b5ff22e6dd0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:10:33.329277 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:10:33.336781 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a919d628-4aa9-4c1f-9415-aeb21c6fdf0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:10:33.337642 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a919d628-4aa9-4c1f-9415-aeb21c6fdf0f None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:33.338020 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a919d628-4aa9-4c1f-9415-aeb21c6fdf0f None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:33.338385 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a919d628-4aa9-4c1f-9415-aeb21c6fdf0f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:33.425790 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 114/221] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 12:10:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 12:10:33.432467 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a919d628-4aa9-4c1f-9415-aeb21c6fdf0f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:10:33.433983 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a919d628-4aa9-4c1f-9415-aeb21c6fdf0f None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:10:33.535680 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 108/222] 10.222.0.15 () {58 vars in 1282 bytes} [Thu Sep 15 12:10:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 15 12:10:33.542014 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386fb385-ca8f-41ab-b361-b5ff22e6dd0a None None] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:10:33.584150 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a919d628-4aa9-4c1f-9415-aeb21c6fdf0f None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:10:33.620015 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:10:33.672161 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386fb385-ca8f-41ab-b361-b5ff22e6dd0a None None] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:10:33.701523 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:10:33.721391 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4514e4ac-b9d1-4a17-adb7-f730d475e630 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:10:33.722428 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4514e4ac-b9d1-4a17-adb7-f730d475e630 None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:33.722790 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a919d628-4aa9-4c1f-9415-aeb21c6fdf0f None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:10:33.723141 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4514e4ac-b9d1-4a17-adb7-f730d475e630 None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:33.723669 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4514e4ac-b9d1-4a17-adb7-f730d475e630 None admin] PATH_INFO: `/v3/projects/bd9c06e0f78e4d26902fd521a2fe013e` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:33.725238 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4514e4ac-b9d1-4a17-adb7-f730d475e630 None admin] RBAC: Authorizing `identity:get_project(project_id=bd9c06e0f78e4d26902fd521a2fe013e)` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:10:33.726929 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4514e4ac-b9d1-4a17-adb7-f730d475e630 None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:10:33.729257 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 115/223] 10.222.0.15 () {56 vars in 1186 bytes} [Thu Sep 15 12:10:33 2022] GET /identity/v3/projects/bd9c06e0f78e4d26902fd521a2fe013e => generated 302 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:10:33.730669 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4514e4ac-b9d1-4a17-adb7-f730d475e630 None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:10:33.765289 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b062452d-557c-44f8-8bc1-243cf9c1efa5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:10:33.765945 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b062452d-557c-44f8-8bc1-243cf9c1efa5 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:33.766227 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b062452d-557c-44f8-8bc1-243cf9c1efa5 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:33.766498 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b062452d-557c-44f8-8bc1-243cf9c1efa5 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:33.780273 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b062452d-557c-44f8-8bc1-243cf9c1efa5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 12:10:33.786568 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application [None req-b062452d-557c-44f8-8bc1-243cf9c1efa5 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 15 12:10:33.786568 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:10:33.786568 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:10:33.786568 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:10:33.786568 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:10:33.786568 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:10:33.786568 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:10:33.786568 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:10:33.786568 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:10:33.786568 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:10:33.786568 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:10:33.786568 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:10:33.786568 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 12:10:33.786568 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 12:10:33.786568 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 12:10:33.786568 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 12:10:33.787485 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 12:10:33.787485 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 12:10:33.787485 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 12:10:33.787485 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 12:10:33.787485 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:10:33.787485 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:10:33.787485 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:10:33.787485 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:10:33.787485 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:10:33.787485 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:10:33.787485 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:10:33.787485 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application with Lock( Sep 15 12:10:33.787485 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:10:33.787485 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:10:33.787485 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:10:33.787485 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:10:33.788317 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:10:33.788317 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:10:33.788317 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:10:33.788317 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:10:33.788317 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 12:10:33.788317 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 12:10:33.788317 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 12:10:33.788317 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 12:10:33.788317 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 12:10:33.788317 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 12:10:33.788317 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 15 12:10:33.788317 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 15 12:10:33.788317 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 109/224] 10.222.0.15 () {56 vars in 1114 bytes} [Thu Sep 15 12:10:33 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 15 12:10:33.789006 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b062452d-557c-44f8-8bc1-243cf9c1efa5 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:10:33.844977 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4514e4ac-b9d1-4a17-adb7-f730d475e630 None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:10:33.880694 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60e5be2-4000-4b83-80c2-d5a914295de0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:10:33.882354 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60e5be2-4000-4b83-80c2-d5a914295de0 None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:33.882838 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60e5be2-4000-4b83-80c2-d5a914295de0 None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:33.883377 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60e5be2-4000-4b83-80c2-d5a914295de0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:33.896649 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60e5be2-4000-4b83-80c2-d5a914295de0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 12:10:33.902913 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application [None req-d60e5be2-4000-4b83-80c2-d5a914295de0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 15 12:10:33.902913 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:10:33.902913 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:10:33.902913 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:10:33.902913 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:10:33.902913 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:10:33.902913 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:10:33.902913 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:10:33.902913 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:10:33.902913 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:10:33.902913 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:10:33.902913 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:10:33.902913 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 12:10:33.902913 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 12:10:33.902913 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 12:10:33.902913 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 12:10:33.903817 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 12:10:33.903817 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 12:10:33.903817 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 12:10:33.903817 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 12:10:33.903817 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:10:33.903817 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:10:33.903817 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:10:33.903817 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:10:33.903817 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:10:33.903817 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:10:33.903817 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:10:33.903817 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application with Lock( Sep 15 12:10:33.903817 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:10:33.903817 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:10:33.903817 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:10:33.903817 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:10:33.904603 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:10:33.904603 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:10:33.904603 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:10:33.904603 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:10:33.904603 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 12:10:33.904603 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 12:10:33.904603 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 12:10:33.904603 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 12:10:33.904603 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 12:10:33.904603 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 12:10:33.904603 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 15 12:10:33.904603 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application Sep 15 12:10:33.907005 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 116/225] 10.222.0.15 () {56 vars in 1096 bytes} [Thu Sep 15 12:10:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:10:33.907999 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60e5be2-4000-4b83-80c2-d5a914295de0 None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:10:33.909718 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:10:33.931652 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b062452d-557c-44f8-8bc1-243cf9c1efa5 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:10:33.962609 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:10:33.976498 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e721e373-121e-494d-b734-afbcc3a04d58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:10:33.977163 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e721e373-121e-494d-b734-afbcc3a04d58 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:33.977439 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e721e373-121e-494d-b734-afbcc3a04d58 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:33.977711 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e721e373-121e-494d-b734-afbcc3a04d58 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:33.978439 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e721e373-121e-494d-b734-afbcc3a04d58 None admin] RBAC: Adding query filter params (role.id=0679d6628a25438d90dc4bdc0401eda2, scope.project.id=bd9c06e0f78e4d26902fd521a2fe013e, user.id=b8b4429c9fff4109b34e2ffafd7ba319) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:10:33.978740 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e721e373-121e-494d-b734-afbcc3a04d58 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:10:33.980460 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e721e373-121e-494d-b734-afbcc3a04d58 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:10:33.986718 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 110/226] 10.222.0.15 () {56 vars in 1374 bytes} [Thu Sep 15 12:10:33 2022] GET /identity/v3/role_assignments?user.id=b8b4429c9fff4109b34e2ffafd7ba319&scope.project.id=bd9c06e0f78e4d26902fd521a2fe013e&role.id=0679d6628a25438d90dc4bdc0401eda2 => generated 259 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:10:33.987254 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e721e373-121e-494d-b734-afbcc3a04d58 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:10:34.094670 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60e5be2-4000-4b83-80c2-d5a914295de0 None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:10:34.160108 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86120e05-f877-445a-90c4-a1415bc5a420 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:10:34.161530 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86120e05-f877-445a-90c4-a1415bc5a420 None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:34.161816 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86120e05-f877-445a-90c4-a1415bc5a420 None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:34.162089 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86120e05-f877-445a-90c4-a1415bc5a420 None admin] PATH_INFO: `/v3/roles` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:34.162757 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86120e05-f877-445a-90c4-a1415bc5a420 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81449) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:10:34.163087 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86120e05-f877-445a-90c4-a1415bc5a420 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:10:34.164470 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86120e05-f877-445a-90c4-a1415bc5a420 None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:10:34.174669 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 117/227] 10.222.0.15 () {56 vars in 1111 bytes} [Thu Sep 15 12:10:33 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:10:34.181185 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86120e05-f877-445a-90c4-a1415bc5a420 None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:10:34.202719 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e721e373-121e-494d-b734-afbcc3a04d58 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:10:34.275677 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c494e5ba-1c86-4119-a30d-2c122ed75b00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:10:34.276308 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c494e5ba-1c86-4119-a30d-2c122ed75b00 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:34.276584 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c494e5ba-1c86-4119-a30d-2c122ed75b00 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:34.277115 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c494e5ba-1c86-4119-a30d-2c122ed75b00 None admin] PATH_INFO: `/v3/roles` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:34.277778 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c494e5ba-1c86-4119-a30d-2c122ed75b00 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:10:34.278865 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c494e5ba-1c86-4119-a30d-2c122ed75b00 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:10:34.280231 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c494e5ba-1c86-4119-a30d-2c122ed75b00 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:10:34.299888 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 111/228] 10.222.0.15 () {56 vars in 1099 bytes} [Thu Sep 15 12:10:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:10:34.310373 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c494e5ba-1c86-4119-a30d-2c122ed75b00 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:10:34.349174 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86120e05-f877-445a-90c4-a1415bc5a420 None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:10:34.350261 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:10:34.366265 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:10:34.403921 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb54425-5967-4721-a5b8-e1a4e06d1bb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:10:34.404560 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb54425-5967-4721-a5b8-e1a4e06d1bb9 None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:34.404863 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb54425-5967-4721-a5b8-e1a4e06d1bb9 None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:34.405134 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb54425-5967-4721-a5b8-e1a4e06d1bb9 None admin] PATH_INFO: `/v3/users/42017b63f1c045cdad77de4fceaa6cb8` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:34.406770 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb54425-5967-4721-a5b8-e1a4e06d1bb9 None admin] RBAC: Authorizing `identity:get_user(user_id=42017b63f1c045cdad77de4fceaa6cb8)` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:10:34.408566 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb54425-5967-4721-a5b8-e1a4e06d1bb9 None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:10:34.410480 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 118/229] 10.222.0.15 () {56 vars in 1177 bytes} [Thu Sep 15 12:10:34 2022] GET /identity/v3/users/42017b63f1c045cdad77de4fceaa6cb8 => generated 283 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 12:10:34.416580 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb54425-5967-4721-a5b8-e1a4e06d1bb9 None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:10:34.524884 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c494e5ba-1c86-4119-a30d-2c122ed75b00 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:10:34.564056 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06bafae6-162f-46cf-983f-bc35dafcd6d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:10:34.565284 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bafae6-162f-46cf-983f-bc35dafcd6d5 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:34.566008 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bafae6-162f-46cf-983f-bc35dafcd6d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:34.566462 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bafae6-162f-46cf-983f-bc35dafcd6d5 None admin] PATH_INFO: `/v3/users/42017b63f1c045cdad77de4fceaa6cb8` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:34.567805 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06bafae6-162f-46cf-983f-bc35dafcd6d5 None admin] RBAC: Authorizing `identity:get_user(user_id=42017b63f1c045cdad77de4fceaa6cb8)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:10:34.571779 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06bafae6-162f-46cf-983f-bc35dafcd6d5 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:10:34.574067 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 112/230] 10.222.0.15 () {56 vars in 1177 bytes} [Thu Sep 15 12:10:34 2022] GET /identity/v3/users/42017b63f1c045cdad77de4fceaa6cb8 => generated 283 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 12:10:34.580791 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06bafae6-162f-46cf-983f-bc35dafcd6d5 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:10:34.596803 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb54425-5967-4721-a5b8-e1a4e06d1bb9 None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:10:34.660409 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45036993-14dd-4362-8270-02f8fb27d7cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:10:34.661124 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45036993-14dd-4362-8270-02f8fb27d7cb None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:34.661336 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45036993-14dd-4362-8270-02f8fb27d7cb None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:34.661600 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45036993-14dd-4362-8270-02f8fb27d7cb None admin] PATH_INFO: `/v3/projects/bd9c06e0f78e4d26902fd521a2fe013e` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:34.664401 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45036993-14dd-4362-8270-02f8fb27d7cb None admin] RBAC: Authorizing `identity:get_project(project_id=bd9c06e0f78e4d26902fd521a2fe013e)` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:10:34.666694 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45036993-14dd-4362-8270-02f8fb27d7cb None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:10:34.668302 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 119/231] 10.222.0.15 () {56 vars in 1186 bytes} [Thu Sep 15 12:10:34 2022] GET /identity/v3/projects/bd9c06e0f78e4d26902fd521a2fe013e => generated 302 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:10:34.685118 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45036993-14dd-4362-8270-02f8fb27d7cb None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:10:34.758463 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:10:34.770433 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:10:34.784826 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06bafae6-162f-46cf-983f-bc35dafcd6d5 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:10:34.850138 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785cc79a-262f-4c17-9e56-bd1ef2d224f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:10:34.850844 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785cc79a-262f-4c17-9e56-bd1ef2d224f4 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:34.851124 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785cc79a-262f-4c17-9e56-bd1ef2d224f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:34.851391 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785cc79a-262f-4c17-9e56-bd1ef2d224f4 None admin] PATH_INFO: `/v3/projects/bd9c06e0f78e4d26902fd521a2fe013e` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:34.855247 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785cc79a-262f-4c17-9e56-bd1ef2d224f4 None admin] RBAC: Authorizing `identity:get_project(project_id=bd9c06e0f78e4d26902fd521a2fe013e)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:10:34.856679 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-785cc79a-262f-4c17-9e56-bd1ef2d224f4 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:10:34.858332 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 113/232] 10.222.0.15 () {56 vars in 1186 bytes} [Thu Sep 15 12:10:34 2022] GET /identity/v3/projects/bd9c06e0f78e4d26902fd521a2fe013e => generated 302 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:10:34.878451 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785cc79a-262f-4c17-9e56-bd1ef2d224f4 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:10:34.898785 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45036993-14dd-4362-8270-02f8fb27d7cb None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:10:34.978336 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8caa2dd9-a528-4d40-a5f7-82f482847d97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:10:34.979013 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8caa2dd9-a528-4d40-a5f7-82f482847d97 None admin] REQUEST_METHOD: `PUT` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:34.979289 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8caa2dd9-a528-4d40-a5f7-82f482847d97 None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:34.979554 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8caa2dd9-a528-4d40-a5f7-82f482847d97 None admin] PATH_INFO: `/v3/projects/bd9c06e0f78e4d26902fd521a2fe013e/users/42017b63f1c045cdad77de4fceaa6cb8/roles/1081b93ddfd547649ba8582f7a6e719e` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:34.991998 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8caa2dd9-a528-4d40-a5f7-82f482847d97 None admin] RBAC: Authorizing `identity:create_grant(project_id=bd9c06e0f78e4d26902fd521a2fe013e, user_id=42017b63f1c045cdad77de4fceaa6cb8, role_id=1081b93ddfd547649ba8582f7a6e719e)` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:10:34.993621 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8caa2dd9-a528-4d40-a5f7-82f482847d97 None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:10:35.024049 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 120/233] 10.222.0.15 () {58 vars in 1439 bytes} [Thu Sep 15 12:10:34 2022] PUT /identity/v3/projects/bd9c06e0f78e4d26902fd521a2fe013e/users/42017b63f1c045cdad77de4fceaa6cb8/roles/1081b93ddfd547649ba8582f7a6e719e => generated 0 bytes in 339 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 12:10:35.095613 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785cc79a-262f-4c17-9e56-bd1ef2d224f4 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:10:35.149722 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e3f7e6-86b6-4291-b7ad-4a6bb722b718 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:10:35.150415 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e3f7e6-86b6-4291-b7ad-4a6bb722b718 None admin] REQUEST_METHOD: `PUT` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:35.150701 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e3f7e6-86b6-4291-b7ad-4a6bb722b718 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:35.151039 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e3f7e6-86b6-4291-b7ad-4a6bb722b718 None admin] PATH_INFO: `/v3/projects/bd9c06e0f78e4d26902fd521a2fe013e/users/42017b63f1c045cdad77de4fceaa6cb8/roles/9c769b5c82884766b2c510b1f0f1364c` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:35.157434 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e3f7e6-86b6-4291-b7ad-4a6bb722b718 None admin] RBAC: Authorizing `identity:create_grant(project_id=bd9c06e0f78e4d26902fd521a2fe013e, user_id=42017b63f1c045cdad77de4fceaa6cb8, role_id=9c769b5c82884766b2c510b1f0f1364c)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:10:35.158929 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e3f7e6-86b6-4291-b7ad-4a6bb722b718 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:10:35.186518 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 114/234] 10.222.0.15 () {58 vars in 1439 bytes} [Thu Sep 15 12:10:34 2022] PUT /identity/v3/projects/bd9c06e0f78e4d26902fd521a2fe013e/users/42017b63f1c045cdad77de4fceaa6cb8/roles/9c769b5c82884766b2c510b1f0f1364c => generated 0 bytes in 308 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 12:10:35.784353 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c6cdaf-bdde-424b-831f-b6b5f80d523c None None] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:35.784923 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c6cdaf-bdde-424b-831f-b6b5f80d523c None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:35.784923 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c6cdaf-bdde-424b-831f-b6b5f80d523c None None] PATH_INFO: `/` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:35.786368 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 121/235] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 12:10:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:10:35.792310 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4eeece8-0223-422c-8ff6-46ead3c7ad72 None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:35.793124 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4eeece8-0223-422c-8ff6-46ead3c7ad72 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:35.793624 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4eeece8-0223-422c-8ff6-46ead3c7ad72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:35.812212 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-b4eeece8-0223-422c-8ff6-46ead3c7ad72 None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:10:35.813777 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-b4eeece8-0223-422c-8ff6-46ead3c7ad72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:10:35.824564 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:10:35.888115 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 115/236] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 12:10:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 12:10:36.100199 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8336c7a8-8ba1-4874-b253-797b62e571af None None] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:36.101349 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8336c7a8-8ba1-4874-b253-797b62e571af None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:36.101818 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8336c7a8-8ba1-4874-b253-797b62e571af None None] PATH_INFO: `/` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:36.103372 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 122/237] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 12:10:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:10:36.108917 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2facb2-e151-4fc8-9938-24bef6f4c22d None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:36.109456 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2facb2-e151-4fc8-9938-24bef6f4c22d None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:36.109926 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2facb2-e151-4fc8-9938-24bef6f4c22d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:36.135631 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-4d2facb2-e151-4fc8-9938-24bef6f4c22d None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:10:36.137179 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-4d2facb2-e151-4fc8-9938-24bef6f4c22d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:10:36.218526 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 116/238] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 12:10:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 12:10:36.243074 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8336c7a8-8ba1-4874-b253-797b62e571af None None] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:10:36.371204 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8336c7a8-8ba1-4874-b253-797b62e571af None None] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:10:36.373449 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:10:36.420402 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca45b7e9-36e6-4252-9c84-d9812575ed34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:10:36.421461 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca45b7e9-36e6-4252-9c84-d9812575ed34 None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:36.421943 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca45b7e9-36e6-4252-9c84-d9812575ed34 None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:36.422440 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca45b7e9-36e6-4252-9c84-d9812575ed34 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:36.440616 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca45b7e9-36e6-4252-9c84-d9812575ed34 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 12:10:36.446070 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application [None req-ca45b7e9-36e6-4252-9c84-d9812575ed34 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 15 12:10:36.446070 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:10:36.446070 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:10:36.446070 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:10:36.446070 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:10:36.446070 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:10:36.446070 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:10:36.446070 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:10:36.446070 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:10:36.446070 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:10:36.446070 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:10:36.446070 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:10:36.446070 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 12:10:36.446070 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 12:10:36.446070 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 12:10:36.446070 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 12:10:36.447042 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 12:10:36.447042 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 12:10:36.447042 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 12:10:36.447042 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 12:10:36.447042 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:10:36.447042 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:10:36.447042 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:10:36.447042 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:10:36.447042 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:10:36.447042 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:10:36.447042 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:10:36.447042 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application with Lock( Sep 15 12:10:36.447042 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:10:36.447042 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:10:36.447042 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:10:36.447042 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:10:36.447839 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:10:36.447839 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:10:36.447839 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:10:36.447839 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:10:36.447839 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 12:10:36.447839 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 12:10:36.447839 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 12:10:36.447839 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 12:10:36.447839 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 12:10:36.447839 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 12:10:36.447839 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 15 12:10:36.447839 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application Sep 15 12:10:36.450412 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 123/239] 10.222.0.15 () {56 vars in 1099 bytes} [Thu Sep 15 12:10:36 2022] GET /identity/v3/roles/reader => generated 84 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:10:36.456445 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2facb2-e151-4fc8-9938-24bef6f4c22d None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:10:36.538936 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca45b7e9-36e6-4252-9c84-d9812575ed34 None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:10:36.559507 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:10:36.586992 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2facb2-e151-4fc8-9938-24bef6f4c22d None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:10:36.624258 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da10c85-cf1f-4469-85c4-5cf38fba09da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:10:36.625242 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da10c85-cf1f-4469-85c4-5cf38fba09da None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:36.625698 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da10c85-cf1f-4469-85c4-5cf38fba09da None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:36.626136 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da10c85-cf1f-4469-85c4-5cf38fba09da None admin] PATH_INFO: `/v3/roles` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:36.626969 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da10c85-cf1f-4469-85c4-5cf38fba09da None admin] RBAC: Adding query filter params (name=reader) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:10:36.627500 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da10c85-cf1f-4469-85c4-5cf38fba09da None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:10:36.629191 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da10c85-cf1f-4469-85c4-5cf38fba09da None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:10:36.638513 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 117/240] 10.222.0.15 () {56 vars in 1101 bytes} [Thu Sep 15 12:10:36 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:10:36.644334 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d60ff8a-bbe1-4be8-95ff-4dc87d9b4b5e None None] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:36.644999 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d60ff8a-bbe1-4be8-95ff-4dc87d9b4b5e None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:36.645591 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d60ff8a-bbe1-4be8-95ff-4dc87d9b4b5e None None] PATH_INFO: `/` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:36.647049 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 118/241] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 12:10:36 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:10:36.647410 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca45b7e9-36e6-4252-9c84-d9812575ed34 None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:10:36.648196 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d60ff8a-bbe1-4be8-95ff-4dc87d9b4b5e None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:10:36.689829 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e448b5-0ee8-4a7a-8e8a-9f3f7a8a2708 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:10:36.690457 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e448b5-0ee8-4a7a-8e8a-9f3f7a8a2708 None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:36.690731 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e448b5-0ee8-4a7a-8e8a-9f3f7a8a2708 None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:36.691175 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e448b5-0ee8-4a7a-8e8a-9f3f7a8a2708 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:36.692781 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e448b5-0ee8-4a7a-8e8a-9f3f7a8a2708 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:10:36.694655 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e448b5-0ee8-4a7a-8e8a-9f3f7a8a2708 None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:10:36.700836 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 124/242] 10.222.0.15 () {56 vars in 1108 bytes} [Thu Sep 15 12:10:36 2022] GET /identity/v3/domains/default => generated 199 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 12:10:36.702440 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b95c76-c39c-4906-8eb3-a2b6087df425 None None] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:36.702718 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b95c76-c39c-4906-8eb3-a2b6087df425 None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:36.702986 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b95c76-c39c-4906-8eb3-a2b6087df425 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:36.721260 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.auth.core [None req-84b95c76-c39c-4906-8eb3-a2b6087df425 None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81449) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:10:36.722763 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-84b95c76-c39c-4906-8eb3-a2b6087df425 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:10:36.748887 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:10:36.770354 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d60ff8a-bbe1-4be8-95ff-4dc87d9b4b5e None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:10:36.796664 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 125/243] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 12:10:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 12:10:36.798267 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b95c76-c39c-4906-8eb3-a2b6087df425 None None] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:10:36.807361 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70fc3a2b-e834-4488-89e3-ac2b95b3c84e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:10:36.808009 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fc3a2b-e834-4488-89e3-ac2b95b3c84e None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:36.808289 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fc3a2b-e834-4488-89e3-ac2b95b3c84e None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:36.808568 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70fc3a2b-e834-4488-89e3-ac2b95b3c84e None admin] PATH_INFO: `/v3/users/23e90deb1aec4e90845c059729c5f57d` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:36.811431 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:10:36.823714 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70fc3a2b-e834-4488-89e3-ac2b95b3c84e None admin] RBAC: Authorizing `identity:get_user(user_id=23e90deb1aec4e90845c059729c5f57d)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:10:36.825228 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70fc3a2b-e834-4488-89e3-ac2b95b3c84e None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:10:36.827273 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 119/244] 10.222.0.15 () {56 vars in 1177 bytes} [Thu Sep 15 12:10:36 2022] GET /identity/v3/users/23e90deb1aec4e90845c059729c5f57d => generated 297 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:10:36.832581 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70fc3a2b-e834-4488-89e3-ac2b95b3c84e None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:10:36.959527 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b95c76-c39c-4906-8eb3-a2b6087df425 None None] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:10:37.005505 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70fc3a2b-e834-4488-89e3-ac2b95b3c84e None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:10:37.042374 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c791122-f512-4faa-8759-2076062f891d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:10:37.042982 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c791122-f512-4faa-8759-2076062f891d None admin] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:37.043259 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c791122-f512-4faa-8759-2076062f891d None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:37.043539 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c791122-f512-4faa-8759-2076062f891d None admin] PATH_INFO: `/v3/users` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:37.044180 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c791122-f512-4faa-8759-2076062f891d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:10:37.045427 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86407d1-c516-47d0-b159-152180f082f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:10:37.045693 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c791122-f512-4faa-8759-2076062f891d None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:10:37.046699 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86407d1-c516-47d0-b159-152180f082f9 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:37.047170 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86407d1-c516-47d0-b159-152180f082f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:37.047626 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86407d1-c516-47d0-b159-152180f082f9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:37.096219 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 126/245] 10.222.0.15 () {60 vars in 1132 bytes} [Thu Sep 15 12:10:36 2022] POST /identity/v3/users => generated 293 bytes in 298 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 12:10:37.097144 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c791122-f512-4faa-8759-2076062f891d None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:10:37.121262 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:10:37.130097 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:10:37.144715 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d86407d1-c516-47d0-b159-152180f082f9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:10:37.151479 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d86407d1-c516-47d0-b159-152180f082f9 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:10:37.267016 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c791122-f512-4faa-8759-2076062f891d None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:10:37.338756 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 120/246] 10.222.0.15 () {58 vars in 1282 bytes} [Thu Sep 15 12:10:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 507 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 15 12:10:37.340160 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac857d6-c0ce-4417-a4c8-9736723c3f98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:10:37.340844 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac857d6-c0ce-4417-a4c8-9736723c3f98 None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:37.341126 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac857d6-c0ce-4417-a4c8-9736723c3f98 None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:37.341389 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac857d6-c0ce-4417-a4c8-9736723c3f98 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:37.346630 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86407d1-c516-47d0-b159-152180f082f9 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:10:37.352867 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac857d6-c0ce-4417-a4c8-9736723c3f98 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 12:10:37.358394 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application [None req-2ac857d6-c0ce-4417-a4c8-9736723c3f98 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 15 12:10:37.358394 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:10:37.358394 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:10:37.358394 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:10:37.358394 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:10:37.358394 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:10:37.358394 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:10:37.358394 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:10:37.358394 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:10:37.358394 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:10:37.358394 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:10:37.358394 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:10:37.358394 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 12:10:37.358394 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 12:10:37.358394 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 12:10:37.358394 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 12:10:37.359275 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 12:10:37.359275 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 12:10:37.359275 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 12:10:37.359275 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 12:10:37.359275 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:10:37.359275 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:10:37.359275 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:10:37.359275 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:10:37.359275 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:10:37.359275 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:10:37.359275 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:10:37.359275 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application with Lock( Sep 15 12:10:37.359275 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:10:37.359275 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:10:37.359275 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:10:37.359275 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:10:37.360078 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:10:37.360078 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:10:37.360078 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:10:37.360078 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:10:37.360078 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 12:10:37.360078 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 12:10:37.360078 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 12:10:37.360078 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 12:10:37.360078 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 12:10:37.360078 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 12:10:37.360078 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 15 12:10:37.360078 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application Sep 15 12:10:37.360078 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 127/247] 10.222.0.15 () {56 vars in 1099 bytes} [Thu Sep 15 12:10:37 2022] GET /identity/v3/roles/member => generated 84 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:10:37.368376 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac857d6-c0ce-4417-a4c8-9736723c3f98 None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:10:37.471121 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:10:37.471884 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:10:37.480870 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86407d1-c516-47d0-b159-152180f082f9 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:10:37.499733 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac857d6-c0ce-4417-a4c8-9736723c3f98 None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:10:37.539681 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b5be3c-bc65-48ce-8196-ab9d2a10b09b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:10:37.540631 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b5be3c-bc65-48ce-8196-ab9d2a10b09b None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:37.540841 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b5be3c-bc65-48ce-8196-ab9d2a10b09b None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:37.541211 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b5be3c-bc65-48ce-8196-ab9d2a10b09b None admin] PATH_INFO: `/v3/roles` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:37.541970 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b5be3c-bc65-48ce-8196-ab9d2a10b09b None admin] RBAC: Adding query filter params (name=member) {{(pid=81449) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:10:37.542387 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b5be3c-bc65-48ce-8196-ab9d2a10b09b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:10:37.543857 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b5be3c-bc65-48ce-8196-ab9d2a10b09b None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:10:37.546241 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc7725a-b74f-4ff4-932f-814dec415a34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:10:37.547042 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc7725a-b74f-4ff4-932f-814dec415a34 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:37.547459 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc7725a-b74f-4ff4-932f-814dec415a34 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:37.547885 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc7725a-b74f-4ff4-932f-814dec415a34 None admin] PATH_INFO: `/v3/projects/bd9c06e0f78e4d26902fd521a2fe013e` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:37.549285 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc7725a-b74f-4ff4-932f-814dec415a34 None admin] RBAC: Authorizing `identity:get_project(project_id=bd9c06e0f78e4d26902fd521a2fe013e)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:10:37.551338 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc7725a-b74f-4ff4-932f-814dec415a34 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:10:37.553233 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 121/248] 10.222.0.15 () {56 vars in 1186 bytes} [Thu Sep 15 12:10:37 2022] GET /identity/v3/projects/bd9c06e0f78e4d26902fd521a2fe013e => generated 302 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:10:37.558645 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 128/249] 10.222.0.15 () {56 vars in 1101 bytes} [Thu Sep 15 12:10:37 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:10:37.559396 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b5be3c-bc65-48ce-8196-ab9d2a10b09b None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:10:37.560608 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc7725a-b74f-4ff4-932f-814dec415a34 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:10:37.686855 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc7725a-b74f-4ff4-932f-814dec415a34 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:10:37.695373 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b5be3c-bc65-48ce-8196-ab9d2a10b09b None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:10:37.728961 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63dba4f-0da2-419a-a776-1fe88f272fd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:10:37.730858 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63dba4f-0da2-419a-a776-1fe88f272fd1 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:37.731442 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63dba4f-0da2-419a-a776-1fe88f272fd1 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:37.731997 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63dba4f-0da2-419a-a776-1fe88f272fd1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:37.733152 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63dba4f-0da2-419a-a776-1fe88f272fd1 None admin] RBAC: Adding query filter params (role.id=1807da0df67b49739d033f8c0e7456f0, scope.project.id=bd9c06e0f78e4d26902fd521a2fe013e, user.id=23e90deb1aec4e90845c059729c5f57d) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:10:37.733630 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63dba4f-0da2-419a-a776-1fe88f272fd1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:10:37.735244 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63dba4f-0da2-419a-a776-1fe88f272fd1 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:10:37.736618 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c8bb37-1dde-47a2-9f17-9d53c357df85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:10:37.737364 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c8bb37-1dde-47a2-9f17-9d53c357df85 None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:37.737734 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c8bb37-1dde-47a2-9f17-9d53c357df85 None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:37.738152 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c8bb37-1dde-47a2-9f17-9d53c357df85 None admin] PATH_INFO: `/v3/users/b8b4429c9fff4109b34e2ffafd7ba319` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:37.739785 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c8bb37-1dde-47a2-9f17-9d53c357df85 None admin] RBAC: Authorizing `identity:get_user(user_id=b8b4429c9fff4109b34e2ffafd7ba319)` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:10:37.741666 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c8bb37-1dde-47a2-9f17-9d53c357df85 None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:10:37.743499 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 129/251] 10.222.0.15 () {56 vars in 1177 bytes} [Thu Sep 15 12:10:37 2022] GET /identity/v3/users/b8b4429c9fff4109b34e2ffafd7ba319 => generated 297 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 12:10:37.743683 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 122/251] 10.222.0.15 () {56 vars in 1374 bytes} [Thu Sep 15 12:10:37 2022] GET /identity/v3/role_assignments?user.id=23e90deb1aec4e90845c059729c5f57d&scope.project.id=bd9c06e0f78e4d26902fd521a2fe013e&role.id=1807da0df67b49739d033f8c0e7456f0 => generated 259 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:10:37.749524 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c8bb37-1dde-47a2-9f17-9d53c357df85 None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:10:37.762586 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:10:37.816567 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb199c4-17e6-4036-aa0b-b6fbe6bb7f91 None None] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:37.817562 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb199c4-17e6-4036-aa0b-b6fbe6bb7f91 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:37.818136 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb199c4-17e6-4036-aa0b-b6fbe6bb7f91 None None] PATH_INFO: `/` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:37.820271 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 123/252] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 12:10:37 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 15 12:10:37.830047 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a41618-9c13-4643-a76f-f224d15897d1 None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:37.830356 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a41618-9c13-4643-a76f-f224d15897d1 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:37.830672 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a41618-9c13-4643-a76f-f224d15897d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:37.861696 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-e3a41618-9c13-4643-a76f-f224d15897d1 None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:10:37.862901 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-e3a41618-9c13-4643-a76f-f224d15897d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:10:37.888325 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:10:37.890826 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c8bb37-1dde-47a2-9f17-9d53c357df85 None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:10:37.937707 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d52c04-a587-413d-83a6-4bcc62a1afd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:10:37.938486 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d52c04-a587-413d-83a6-4bcc62a1afd1 None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:37.938993 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d52c04-a587-413d-83a6-4bcc62a1afd1 None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:37.939284 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d52c04-a587-413d-83a6-4bcc62a1afd1 None admin] PATH_INFO: `/v3/projects/bd9c06e0f78e4d26902fd521a2fe013e` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:37.941898 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d52c04-a587-413d-83a6-4bcc62a1afd1 None admin] RBAC: Authorizing `identity:get_project(project_id=bd9c06e0f78e4d26902fd521a2fe013e)` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:10:37.943747 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d52c04-a587-413d-83a6-4bcc62a1afd1 None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:10:37.946762 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 130/253] 10.222.0.15 () {56 vars in 1186 bytes} [Thu Sep 15 12:10:37 2022] GET /identity/v3/projects/bd9c06e0f78e4d26902fd521a2fe013e => generated 302 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:10:37.951120 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 124/254] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 12:10:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 12:10:37.956410 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d52c04-a587-413d-83a6-4bcc62a1afd1 None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:10:37.962672 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231e77de-1843-4833-9282-e94fe395c389 None None] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:37.962962 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231e77de-1843-4833-9282-e94fe395c389 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:37.963216 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231e77de-1843-4833-9282-e94fe395c389 None None] PATH_INFO: `/` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:37.965086 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 125/255] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 12:10:37 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:10:37.981591 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f432260e-bb3d-480e-badf-f8a34cbd1b0f None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:37.981924 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f432260e-bb3d-480e-badf-f8a34cbd1b0f None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:37.982196 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f432260e-bb3d-480e-badf-f8a34cbd1b0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:38.009990 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-f432260e-bb3d-480e-badf-f8a34cbd1b0f None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:10:38.012606 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-f432260e-bb3d-480e-badf-f8a34cbd1b0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:10:38.082065 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:10:38.095044 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 126/256] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 12:10:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 15 12:10:38.117052 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d52c04-a587-413d-83a6-4bcc62a1afd1 None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:10:38.161128 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3063e4-3a25-4ce2-887f-3bd2cb0c1bad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:10:38.162100 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3063e4-3a25-4ce2-887f-3bd2cb0c1bad None admin] REQUEST_METHOD: `PUT` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:38.162555 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3063e4-3a25-4ce2-887f-3bd2cb0c1bad None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:38.163478 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3063e4-3a25-4ce2-887f-3bd2cb0c1bad None admin] PATH_INFO: `/v3/projects/bd9c06e0f78e4d26902fd521a2fe013e/users/b8b4429c9fff4109b34e2ffafd7ba319/roles/0679d6628a25438d90dc4bdc0401eda2` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:38.170127 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3063e4-3a25-4ce2-887f-3bd2cb0c1bad None admin] RBAC: Authorizing `identity:create_grant(project_id=bd9c06e0f78e4d26902fd521a2fe013e, user_id=b8b4429c9fff4109b34e2ffafd7ba319, role_id=0679d6628a25438d90dc4bdc0401eda2)` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:10:38.171743 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3063e4-3a25-4ce2-887f-3bd2cb0c1bad None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:10:38.192403 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 131/257] 10.222.0.15 () {58 vars in 1439 bytes} [Thu Sep 15 12:10:37 2022] PUT /identity/v3/projects/bd9c06e0f78e4d26902fd521a2fe013e/users/b8b4429c9fff4109b34e2ffafd7ba319/roles/0679d6628a25438d90dc4bdc0401eda2 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:10:38.398492 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f432260e-bb3d-480e-badf-f8a34cbd1b0f None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:10:38.466533 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3063e4-3a25-4ce2-887f-3bd2cb0c1bad None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:10:38.530514 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:10:38.558553 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f432260e-bb3d-480e-badf-f8a34cbd1b0f None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:10:38.608227 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d9473b-0eb2-4f46-a5d5-1aa3a0cbb817 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:10:38.608911 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d9473b-0eb2-4f46-a5d5-1aa3a0cbb817 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:38.609185 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d9473b-0eb2-4f46-a5d5-1aa3a0cbb817 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:38.609451 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d9473b-0eb2-4f46-a5d5-1aa3a0cbb817 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:38.622096 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d9473b-0eb2-4f46-a5d5-1aa3a0cbb817 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 12:10:38.627231 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application [None req-d4d9473b-0eb2-4f46-a5d5-1aa3a0cbb817 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 15 12:10:38.627231 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:10:38.627231 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:10:38.627231 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:10:38.627231 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:10:38.627231 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:10:38.627231 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:10:38.627231 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:10:38.627231 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:10:38.627231 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:10:38.627231 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:10:38.627231 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:10:38.627231 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 12:10:38.627231 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 12:10:38.627231 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 12:10:38.627231 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 12:10:38.628135 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 12:10:38.628135 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 12:10:38.628135 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 12:10:38.628135 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 12:10:38.628135 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:10:38.628135 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:10:38.628135 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:10:38.628135 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:10:38.628135 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:10:38.628135 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:10:38.628135 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:10:38.628135 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application with Lock( Sep 15 12:10:38.628135 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:10:38.628135 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:10:38.628135 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:10:38.628135 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:10:38.628966 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:10:38.628966 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:10:38.628966 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:10:38.628966 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:10:38.628966 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 12:10:38.628966 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 12:10:38.628966 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 12:10:38.628966 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 12:10:38.628966 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 12:10:38.628966 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 12:10:38.628966 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 15 12:10:38.628966 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 15 12:10:38.628966 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 127/258] 10.222.0.15 () {56 vars in 1114 bytes} [Thu Sep 15 12:10:38 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:10:38.636699 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d9473b-0eb2-4f46-a5d5-1aa3a0cbb817 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:10:38.668617 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3063e4-3a25-4ce2-887f-3bd2cb0c1bad None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:10:38.746417 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aaa175f-fcb8-465e-a716-ba741f673b28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:10:38.747070 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aaa175f-fcb8-465e-a716-ba741f673b28 None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:38.747350 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aaa175f-fcb8-465e-a716-ba741f673b28 None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:38.747619 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aaa175f-fcb8-465e-a716-ba741f673b28 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:38.754620 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:10:38.769940 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aaa175f-fcb8-465e-a716-ba741f673b28 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 12:10:38.775748 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application [None req-0aaa175f-fcb8-465e-a716-ba741f673b28 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 15 12:10:38.775748 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:10:38.775748 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:10:38.775748 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:10:38.775748 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:10:38.775748 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:10:38.775748 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:10:38.775748 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:10:38.775748 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:10:38.775748 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:10:38.775748 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:10:38.775748 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:10:38.775748 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 12:10:38.775748 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 12:10:38.775748 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 12:10:38.775748 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 12:10:38.776672 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 12:10:38.776672 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 12:10:38.776672 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 12:10:38.776672 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 12:10:38.776672 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:10:38.776672 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:10:38.776672 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:10:38.776672 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:10:38.776672 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:10:38.776672 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:10:38.776672 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:10:38.776672 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application with Lock( Sep 15 12:10:38.776672 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:10:38.776672 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:10:38.776672 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:10:38.776672 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:10:38.778357 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:10:38.778357 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:10:38.778357 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:10:38.778357 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:10:38.778357 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 12:10:38.778357 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 12:10:38.778357 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 12:10:38.778357 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 12:10:38.778357 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 12:10:38.778357 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 12:10:38.778357 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 15 12:10:38.778357 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application Sep 15 12:10:38.778357 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 132/259] 10.222.0.15 () {56 vars in 1096 bytes} [Thu Sep 15 12:10:38 2022] GET /identity/v3/roles/admin => generated 83 bytes in 311 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 15 12:10:38.786495 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aaa175f-fcb8-465e-a716-ba741f673b28 None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:10:38.838907 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d9473b-0eb2-4f46-a5d5-1aa3a0cbb817 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:10:38.929778 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d0a9bf-549e-45f1-b9a4-a65dce4dee38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:10:38.930319 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d0a9bf-549e-45f1-b9a4-a65dce4dee38 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:38.930595 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d0a9bf-549e-45f1-b9a4-a65dce4dee38 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:38.930864 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d0a9bf-549e-45f1-b9a4-a65dce4dee38 None admin] PATH_INFO: `/v3/roles` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:38.931537 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d0a9bf-549e-45f1-b9a4-a65dce4dee38 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:10:38.931834 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d0a9bf-549e-45f1-b9a4-a65dce4dee38 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:10:38.933256 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d0a9bf-549e-45f1-b9a4-a65dce4dee38 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:10:38.947769 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 128/260] 10.222.0.15 () {56 vars in 1111 bytes} [Thu Sep 15 12:10:38 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 311 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:10:38.962133 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d0a9bf-549e-45f1-b9a4-a65dce4dee38 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:10:38.986819 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:10:38.989061 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aaa175f-fcb8-465e-a716-ba741f673b28 None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:10:39.058819 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350a8462-3fe1-42f8-a1f4-bbe699da7b41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:10:39.059446 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350a8462-3fe1-42f8-a1f4-bbe699da7b41 None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:39.059728 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350a8462-3fe1-42f8-a1f4-bbe699da7b41 None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:39.060001 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350a8462-3fe1-42f8-a1f4-bbe699da7b41 None admin] PATH_INFO: `/v3/roles` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:39.060676 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-350a8462-3fe1-42f8-a1f4-bbe699da7b41 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81449) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:10:39.061104 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-350a8462-3fe1-42f8-a1f4-bbe699da7b41 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:10:39.062408 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-350a8462-3fe1-42f8-a1f4-bbe699da7b41 None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:10:39.076682 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 133/261] 10.222.0.15 () {56 vars in 1099 bytes} [Thu Sep 15 12:10:38 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:10:39.082684 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350a8462-3fe1-42f8-a1f4-bbe699da7b41 None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:10:39.196013 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:10:39.216421 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d0a9bf-549e-45f1-b9a4-a65dce4dee38 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:10:39.258835 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350a8462-3fe1-42f8-a1f4-bbe699da7b41 None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:10:39.296540 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19dcf64-748f-4605-90fb-b448fc61b6e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:10:39.297441 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19dcf64-748f-4605-90fb-b448fc61b6e2 None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:39.297722 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19dcf64-748f-4605-90fb-b448fc61b6e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:39.297987 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19dcf64-748f-4605-90fb-b448fc61b6e2 None admin] PATH_INFO: `/v3/users/42017b63f1c045cdad77de4fceaa6cb8` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:39.299122 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a19dcf64-748f-4605-90fb-b448fc61b6e2 None admin] RBAC: Authorizing `identity:get_user(user_id=42017b63f1c045cdad77de4fceaa6cb8)` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:10:39.300997 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a19dcf64-748f-4605-90fb-b448fc61b6e2 None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:10:39.302869 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 134/262] 10.222.0.15 () {56 vars in 1177 bytes} [Thu Sep 15 12:10:39 2022] GET /identity/v3/users/42017b63f1c045cdad77de4fceaa6cb8 => generated 283 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 12:10:39.308682 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19dcf64-748f-4605-90fb-b448fc61b6e2 None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:10:39.320485 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4b75f1-e212-492b-a54f-1cc68ffb5951 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:10:39.321173 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4b75f1-e212-492b-a54f-1cc68ffb5951 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:39.321451 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4b75f1-e212-492b-a54f-1cc68ffb5951 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:39.321721 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f4b75f1-e212-492b-a54f-1cc68ffb5951 None admin] PATH_INFO: `/v3/users/42017b63f1c045cdad77de4fceaa6cb8` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:39.322852 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4b75f1-e212-492b-a54f-1cc68ffb5951 None admin] RBAC: Authorizing `identity:get_user(user_id=42017b63f1c045cdad77de4fceaa6cb8)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:10:39.326033 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f4b75f1-e212-492b-a54f-1cc68ffb5951 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:10:39.327600 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 129/263] 10.222.0.15 () {56 vars in 1177 bytes} [Thu Sep 15 12:10:38 2022] GET /identity/v3/users/42017b63f1c045cdad77de4fceaa6cb8 => generated 283 bytes in 366 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:10:39.336228 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4b75f1-e212-492b-a54f-1cc68ffb5951 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:10:39.486024 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:10:39.502803 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19dcf64-748f-4605-90fb-b448fc61b6e2 None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:10:39.511967 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f4b75f1-e212-492b-a54f-1cc68ffb5951 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:10:39.545845 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:10:39.547732 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7316837-a711-4851-b741-79ae172897bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:10:39.548566 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7316837-a711-4851-b741-79ae172897bc None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:39.549070 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7316837-a711-4851-b741-79ae172897bc None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:39.549564 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7316837-a711-4851-b741-79ae172897bc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:39.564293 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3faac0aa-4a61-49d1-b506-3bbb4d0e2a12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:10:39.565161 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3faac0aa-4a61-49d1-b506-3bbb4d0e2a12 None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:39.565588 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3faac0aa-4a61-49d1-b506-3bbb4d0e2a12 None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:39.566004 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3faac0aa-4a61-49d1-b506-3bbb4d0e2a12 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:39.625275 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7316837-a711-4851-b741-79ae172897bc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:10:39.626823 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7316837-a711-4851-b741-79ae172897bc None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:10:39.710779 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:10:39.737412 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 130/264] 10.222.0.15 () {58 vars in 1282 bytes} [Thu Sep 15 12:10:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 403 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 15 12:10:39.737758 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3faac0aa-4a61-49d1-b506-3bbb4d0e2a12 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:10:39.739231 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3faac0aa-4a61-49d1-b506-3bbb4d0e2a12 None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:10:39.752381 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7316837-a711-4851-b741-79ae172897bc None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:10:39.911834 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7316837-a711-4851-b741-79ae172897bc None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:10:39.995659 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d6531b-9c56-4917-a957-09e296b91e07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:10:39.996241 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d6531b-9c56-4917-a957-09e296b91e07 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:39.996523 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d6531b-9c56-4917-a957-09e296b91e07 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:39.996812 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d6531b-9c56-4917-a957-09e296b91e07 None admin] PATH_INFO: `/v3/projects/bd9c06e0f78e4d26902fd521a2fe013e` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:40.001851 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d6531b-9c56-4917-a957-09e296b91e07 None admin] RBAC: Authorizing `identity:get_project(project_id=bd9c06e0f78e4d26902fd521a2fe013e)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:10:40.003315 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0d6531b-9c56-4917-a957-09e296b91e07 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:10:40.005290 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 131/265] 10.222.0.15 () {56 vars in 1186 bytes} [Thu Sep 15 12:10:39 2022] GET /identity/v3/projects/bd9c06e0f78e4d26902fd521a2fe013e => generated 302 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 12:10:40.011265 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d6531b-9c56-4917-a957-09e296b91e07 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:10:40.031734 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 135/266] 10.222.0.15 () {58 vars in 1282 bytes} [Thu Sep 15 12:10:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 723 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 15 12:10:40.037540 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3faac0aa-4a61-49d1-b506-3bbb4d0e2a12 None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:10:40.086905 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:10:40.106975 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:10:40.226166 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3faac0aa-4a61-49d1-b506-3bbb4d0e2a12 None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:10:40.235313 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d6531b-9c56-4917-a957-09e296b91e07 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:10:40.300990 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dba0577-1381-4386-bee5-0205bd2ee6a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:10:40.301549 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dba0577-1381-4386-bee5-0205bd2ee6a9 None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:40.301823 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dba0577-1381-4386-bee5-0205bd2ee6a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:40.302092 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dba0577-1381-4386-bee5-0205bd2ee6a9 None admin] PATH_INFO: `/v3/projects/bd9c06e0f78e4d26902fd521a2fe013e` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:40.305840 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dba0577-1381-4386-bee5-0205bd2ee6a9 None admin] RBAC: Authorizing `identity:get_project(project_id=bd9c06e0f78e4d26902fd521a2fe013e)` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:10:40.307292 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dba0577-1381-4386-bee5-0205bd2ee6a9 None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:10:40.309253 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0855fa31-41d5-42da-8068-3c3ace621432 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:10:40.309958 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0855fa31-41d5-42da-8068-3c3ace621432 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:40.310235 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0855fa31-41d5-42da-8068-3c3ace621432 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:40.310498 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0855fa31-41d5-42da-8068-3c3ace621432 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:40.311230 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0855fa31-41d5-42da-8068-3c3ace621432 None admin] RBAC: Adding query filter params (role.id=1081b93ddfd547649ba8582f7a6e719e, scope.project.id=bd9c06e0f78e4d26902fd521a2fe013e, user.id=42017b63f1c045cdad77de4fceaa6cb8) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:10:40.311531 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0855fa31-41d5-42da-8068-3c3ace621432 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:10:40.312967 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0855fa31-41d5-42da-8068-3c3ace621432 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:10:40.313570 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 136/267] 10.222.0.15 () {56 vars in 1186 bytes} [Thu Sep 15 12:10:40 2022] GET /identity/v3/projects/bd9c06e0f78e4d26902fd521a2fe013e => generated 302 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:10:40.319980 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dba0577-1381-4386-bee5-0205bd2ee6a9 None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:10:40.329708 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 132/268] 10.222.0.15 () {56 vars in 1374 bytes} [Thu Sep 15 12:10:40 2022] GET /identity/v3/role_assignments?user.id=42017b63f1c045cdad77de4fceaa6cb8&scope.project.id=bd9c06e0f78e4d26902fd521a2fe013e&role.id=1081b93ddfd547649ba8582f7a6e719e => generated 608 bytes in 319 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 12:10:40.449887 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:10:40.459245 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dba0577-1381-4386-bee5-0205bd2ee6a9 None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:10:40.512952 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2ee407-a2ad-4b20-ab07-a37707f7fe2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:10:40.513583 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2ee407-a2ad-4b20-ab07-a37707f7fe2e None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:40.513866 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2ee407-a2ad-4b20-ab07-a37707f7fe2e None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:40.514140 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2ee407-a2ad-4b20-ab07-a37707f7fe2e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:40.514877 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2ee407-a2ad-4b20-ab07-a37707f7fe2e None admin] RBAC: Adding query filter params (role.id=9c769b5c82884766b2c510b1f0f1364c, scope.project.id=bd9c06e0f78e4d26902fd521a2fe013e, user.id=42017b63f1c045cdad77de4fceaa6cb8) {{(pid=81449) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:10:40.515187 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2ee407-a2ad-4b20-ab07-a37707f7fe2e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:10:40.516660 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2ee407-a2ad-4b20-ab07-a37707f7fe2e None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:10:40.524280 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 137/269] 10.222.0.15 () {56 vars in 1374 bytes} [Thu Sep 15 12:10:40 2022] GET /identity/v3/role_assignments?user.id=42017b63f1c045cdad77de4fceaa6cb8&scope.project.id=bd9c06e0f78e4d26902fd521a2fe013e&role.id=9c769b5c82884766b2c510b1f0f1364c => generated 608 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 12:10:40.863455 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c601a9-ef9e-4831-8270-0b10d1f8c810 None None] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:40.863975 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c601a9-ef9e-4831-8270-0b10d1f8c810 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:40.864430 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c601a9-ef9e-4831-8270-0b10d1f8c810 None None] PATH_INFO: `/` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:40.866050 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 133/270] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 12:10:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:10:40.871472 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b2433e-bfb2-4296-84a0-0c7a1691cc23 None None] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:40.871762 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b2433e-bfb2-4296-84a0-0c7a1691cc23 None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:40.872033 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b2433e-bfb2-4296-84a0-0c7a1691cc23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:40.883083 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c0c5cd-6aa4-4fa3-9485-db48ebb217a2 None None] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:40.883379 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c0c5cd-6aa4-4fa3-9485-db48ebb217a2 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:40.883678 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c0c5cd-6aa4-4fa3-9485-db48ebb217a2 None None] PATH_INFO: `/` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:40.885002 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 134/271] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 12:10:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 15 12:10:40.892090 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad042e9a-8954-4a02-a669-757504199c69 None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:40.892721 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad042e9a-8954-4a02-a669-757504199c69 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:40.893348 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad042e9a-8954-4a02-a669-757504199c69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:40.894923 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.auth.core [None req-e2b2433e-bfb2-4296-84a0-0c7a1691cc23 None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81449) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:10:40.896131 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-e2b2433e-bfb2-4296-84a0-0c7a1691cc23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:10:40.914292 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-ad042e9a-8954-4a02-a669-757504199c69 None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:10:40.916321 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-ad042e9a-8954-4a02-a669-757504199c69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:10:40.999652 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 135/272] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 12:10:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 15 12:10:41.007462 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 138/273] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 12:10:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 12:10:41.078015 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6a6e69-ab6f-4853-b921-1d0c15597000 None None] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:41.078720 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6a6e69-ab6f-4853-b921-1d0c15597000 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:41.079342 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6a6e69-ab6f-4853-b921-1d0c15597000 None None] PATH_INFO: `/` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:41.081929 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 136/274] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 12:10:41 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:10:41.088720 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae57c7d-0d4b-4d2e-9c7b-cb141a49e61e None None] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:41.089307 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae57c7d-0d4b-4d2e-9c7b-cb141a49e61e None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:41.089307 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae57c7d-0d4b-4d2e-9c7b-cb141a49e61e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:41.113617 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.auth.core [None req-fae57c7d-0d4b-4d2e-9c7b-cb141a49e61e None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81449) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:10:41.114783 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-fae57c7d-0d4b-4d2e-9c7b-cb141a49e61e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:10:41.188900 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 139/275] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 12:10:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 12:10:41.307357 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6a6e69-ab6f-4853-b921-1d0c15597000 None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:10:41.317767 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae57c7d-0d4b-4d2e-9c7b-cb141a49e61e None None] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:10:41.336942 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:10:41.355633 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:10:41.427380 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6a6e69-ab6f-4853-b921-1d0c15597000 None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:10:41.453503 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae57c7d-0d4b-4d2e-9c7b-cb141a49e61e None None] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:10:41.463409 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0753b60-a0a0-44ed-a26b-46a8a847f2b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:10:41.464050 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0753b60-a0a0-44ed-a26b-46a8a847f2b1 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:41.464327 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0753b60-a0a0-44ed-a26b-46a8a847f2b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:41.464592 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0753b60-a0a0-44ed-a26b-46a8a847f2b1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:41.465705 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0753b60-a0a0-44ed-a26b-46a8a847f2b1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:10:41.467426 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0753b60-a0a0-44ed-a26b-46a8a847f2b1 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:10:41.469256 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 137/276] 10.222.0.15 () {56 vars in 1108 bytes} [Thu Sep 15 12:10:41 2022] GET /identity/v3/domains/default => generated 199 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:10:41.470117 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ccaf3e-ec31-4f84-b902-c6e058f2f319 None None] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:41.470391 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ccaf3e-ec31-4f84-b902-c6e058f2f319 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:41.470662 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ccaf3e-ec31-4f84-b902-c6e058f2f319 None None] PATH_INFO: `/` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:41.471841 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 138/277] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 12:10:41 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:10:41.472532 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ccaf3e-ec31-4f84-b902-c6e058f2f319 None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:10:41.491636 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04def191-f607-40df-bc8c-2db6b692e7f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:10:41.492273 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04def191-f607-40df-bc8c-2db6b692e7f7 None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:41.492552 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04def191-f607-40df-bc8c-2db6b692e7f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:41.492864 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04def191-f607-40df-bc8c-2db6b692e7f7 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:41.502827 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04def191-f607-40df-bc8c-2db6b692e7f7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 12:10:41.507669 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application [None req-04def191-f607-40df-bc8c-2db6b692e7f7 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 15 12:10:41.507669 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:10:41.507669 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:10:41.507669 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:10:41.507669 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:10:41.507669 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:10:41.507669 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:10:41.507669 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:10:41.507669 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:10:41.507669 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:10:41.507669 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:10:41.507669 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:10:41.507669 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 12:10:41.507669 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 12:10:41.507669 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 12:10:41.507669 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 12:10:41.508807 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 12:10:41.508807 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 12:10:41.508807 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 12:10:41.508807 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 12:10:41.508807 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:10:41.508807 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:10:41.508807 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:10:41.508807 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:10:41.508807 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:10:41.508807 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:10:41.508807 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:10:41.508807 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application with Lock( Sep 15 12:10:41.508807 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:10:41.508807 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:10:41.508807 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:10:41.508807 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:10:41.509750 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:10:41.509750 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:10:41.509750 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:10:41.509750 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:10:41.509750 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 12:10:41.509750 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 12:10:41.509750 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 12:10:41.509750 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 12:10:41.509750 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 12:10:41.509750 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 12:10:41.509750 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 15 12:10:41.509750 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application Sep 15 12:10:41.509750 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 140/278] 10.222.0.15 () {56 vars in 1099 bytes} [Thu Sep 15 12:10:41 2022] GET /identity/v3/roles/member => generated 84 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:10:41.514244 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14c8f10-d0c0-49cb-8af4-ee8aa4f08005 None None] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:41.514531 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14c8f10-d0c0-49cb-8af4-ee8aa4f08005 None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:41.514809 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14c8f10-d0c0-49cb-8af4-ee8aa4f08005 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:41.532400 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.auth.core [None req-c14c8f10-d0c0-49cb-8af4-ee8aa4f08005 None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81449) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:10:41.533625 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-c14c8f10-d0c0-49cb-8af4-ee8aa4f08005 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:10:41.576263 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ccaf3e-ec31-4f84-b902-c6e058f2f319 None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:10:41.583627 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:10:41.599208 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 141/279] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 12:10:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 12:10:41.600009 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14c8f10-d0c0-49cb-8af4-ee8aa4f08005 None None] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:10:41.610706 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4408f01-980f-47d2-9463-67e39c6736d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:10:41.611342 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4408f01-980f-47d2-9463-67e39c6736d5 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:41.611620 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4408f01-980f-47d2-9463-67e39c6736d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:41.611890 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4408f01-980f-47d2-9463-67e39c6736d5 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:41.618022 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:10:41.622989 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4408f01-980f-47d2-9463-67e39c6736d5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 12:10:41.627884 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application [None req-a4408f01-980f-47d2-9463-67e39c6736d5 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 15 12:10:41.627884 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:10:41.627884 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:10:41.627884 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:10:41.627884 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:10:41.627884 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:10:41.627884 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:10:41.627884 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:10:41.627884 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:10:41.627884 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:10:41.627884 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:10:41.627884 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:10:41.627884 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 12:10:41.627884 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 12:10:41.627884 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 12:10:41.627884 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 12:10:41.628788 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 12:10:41.628788 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 12:10:41.628788 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 12:10:41.628788 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 12:10:41.628788 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:10:41.628788 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:10:41.628788 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:10:41.628788 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:10:41.628788 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:10:41.628788 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:10:41.628788 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:10:41.628788 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application with Lock( Sep 15 12:10:41.628788 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:10:41.628788 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:10:41.628788 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:10:41.628788 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:10:41.629826 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:10:41.629826 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:10:41.629826 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:10:41.629826 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:10:41.629826 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 12:10:41.629826 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 12:10:41.629826 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 12:10:41.629826 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 12:10:41.629826 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 12:10:41.629826 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 12:10:41.629826 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 15 12:10:41.629826 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 15 12:10:41.629826 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 139/280] 10.222.0.15 () {56 vars in 1099 bytes} [Thu Sep 15 12:10:41 2022] GET /identity/v3/roles/reader => generated 84 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:10:41.630616 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4408f01-980f-47d2-9463-67e39c6736d5 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:10:41.716571 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14c8f10-d0c0-49cb-8af4-ee8aa4f08005 None None] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:10:41.732640 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4408f01-980f-47d2-9463-67e39c6736d5 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:10:41.750489 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2a9740-b0f8-46f0-9136-ccf19cbd763f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:10:41.751098 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2a9740-b0f8-46f0-9136-ccf19cbd763f None admin] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:41.751379 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2a9740-b0f8-46f0-9136-ccf19cbd763f None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:41.751710 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2a9740-b0f8-46f0-9136-ccf19cbd763f None admin] PATH_INFO: `/v3/users` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:41.752357 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2a9740-b0f8-46f0-9136-ccf19cbd763f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:10:41.753825 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2a9740-b0f8-46f0-9136-ccf19cbd763f None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:10:41.769983 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58b7ce2-d680-4b56-a02c-0e135737a7a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:10:41.770866 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58b7ce2-d680-4b56-a02c-0e135737a7a6 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:41.771156 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58b7ce2-d680-4b56-a02c-0e135737a7a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:41.771470 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58b7ce2-d680-4b56-a02c-0e135737a7a6 None admin] PATH_INFO: `/v3/roles` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:41.772163 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58b7ce2-d680-4b56-a02c-0e135737a7a6 None admin] RBAC: Adding query filter params (name=member) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:10:41.772532 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58b7ce2-d680-4b56-a02c-0e135737a7a6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:10:41.774808 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58b7ce2-d680-4b56-a02c-0e135737a7a6 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:10:41.778103 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 142/281] 10.222.0.15 () {60 vars in 1132 bytes} [Thu Sep 15 12:10:41 2022] POST /identity/v3/users => generated 293 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 12:10:41.779055 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2a9740-b0f8-46f0-9136-ccf19cbd763f None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:10:41.783662 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 140/282] 10.222.0.15 () {56 vars in 1101 bytes} [Thu Sep 15 12:10:41 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:10:41.790212 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58b7ce2-d680-4b56-a02c-0e135737a7a6 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:10:41.822557 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:10:41.866655 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:10:41.903676 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58b7ce2-d680-4b56-a02c-0e135737a7a6 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:10:41.914172 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2a9740-b0f8-46f0-9136-ccf19cbd763f None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:10:41.941050 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3e1d1e-541b-4c97-aae2-a103b0a0257c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:10:41.941740 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3e1d1e-541b-4c97-aae2-a103b0a0257c None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:41.941977 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3e1d1e-541b-4c97-aae2-a103b0a0257c None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:41.942250 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3e1d1e-541b-4c97-aae2-a103b0a0257c None admin] PATH_INFO: `/v3/users/2f30ab0c797045a5bdc70b4e0b90a87e` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:41.952729 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c650dda9-5218-4f12-b74a-8ed122de0b40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:10:41.953706 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c650dda9-5218-4f12-b74a-8ed122de0b40 None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:41.954245 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c650dda9-5218-4f12-b74a-8ed122de0b40 None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:41.954521 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c650dda9-5218-4f12-b74a-8ed122de0b40 None admin] PATH_INFO: `/v3/roles` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:41.955201 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c650dda9-5218-4f12-b74a-8ed122de0b40 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81449) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:10:41.955505 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c650dda9-5218-4f12-b74a-8ed122de0b40 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:10:41.956952 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c650dda9-5218-4f12-b74a-8ed122de0b40 None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:10:41.959463 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd3e1d1e-541b-4c97-aae2-a103b0a0257c None admin] RBAC: Authorizing `identity:get_user(user_id=2f30ab0c797045a5bdc70b4e0b90a87e)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:10:41.961210 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd3e1d1e-541b-4c97-aae2-a103b0a0257c None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:10:41.962895 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 141/283] 10.222.0.15 () {56 vars in 1177 bytes} [Thu Sep 15 12:10:41 2022] GET /identity/v3/users/2f30ab0c797045a5bdc70b4e0b90a87e => generated 293 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:10:41.963668 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3e1d1e-541b-4c97-aae2-a103b0a0257c None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:10:41.970977 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 143/284] 10.222.0.15 () {56 vars in 1101 bytes} [Thu Sep 15 12:10:41 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:10:41.971746 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c650dda9-5218-4f12-b74a-8ed122de0b40 None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:10:42.070107 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:10:42.087850 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3e1d1e-541b-4c97-aae2-a103b0a0257c None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:10:42.088238 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c650dda9-5218-4f12-b74a-8ed122de0b40 None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:10:42.117321 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:10:42.130994 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326dccd4-8832-4eda-8ff5-1485c60d8d8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:10:42.131645 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326dccd4-8832-4eda-8ff5-1485c60d8d8c None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:42.131977 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326dccd4-8832-4eda-8ff5-1485c60d8d8c None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:42.132197 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326dccd4-8832-4eda-8ff5-1485c60d8d8c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:42.132980 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326dccd4-8832-4eda-8ff5-1485c60d8d8c None admin] RBAC: Adding query filter params (role.id=0679d6628a25438d90dc4bdc0401eda2, user.id=2f30ab0c797045a5bdc70b4e0b90a87e, scope.system=all) {{(pid=81449) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:10:42.133271 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326dccd4-8832-4eda-8ff5-1485c60d8d8c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:10:42.133369 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4413aa07-35d3-47cc-a673-6deec8ac2bcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:10:42.134030 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4413aa07-35d3-47cc-a673-6deec8ac2bcd None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:42.134380 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4413aa07-35d3-47cc-a673-6deec8ac2bcd None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:42.134655 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4413aa07-35d3-47cc-a673-6deec8ac2bcd None admin] PATH_INFO: `/v3/roles/member` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:42.134749 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326dccd4-8832-4eda-8ff5-1485c60d8d8c None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:10:42.142088 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 144/285] 10.222.0.15 () {56 vars in 1308 bytes} [Thu Sep 15 12:10:41 2022] GET /identity/v3/role_assignments?user.id=2f30ab0c797045a5bdc70b4e0b90a87e&scope.system=all&role.id=0679d6628a25438d90dc4bdc0401eda2 => generated 226 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:10:42.143217 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326dccd4-8832-4eda-8ff5-1485c60d8d8c None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:10:42.152860 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4413aa07-35d3-47cc-a673-6deec8ac2bcd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 12:10:42.159578 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application [None req-4413aa07-35d3-47cc-a673-6deec8ac2bcd None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 15 12:10:42.159578 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:10:42.159578 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:10:42.159578 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:10:42.159578 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:10:42.159578 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:10:42.159578 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:10:42.159578 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:10:42.159578 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:10:42.159578 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:10:42.159578 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:10:42.159578 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:10:42.159578 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 12:10:42.159578 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 12:10:42.159578 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 12:10:42.159578 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 12:10:42.160563 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 12:10:42.160563 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 12:10:42.160563 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 12:10:42.160563 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 12:10:42.160563 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:10:42.160563 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:10:42.160563 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:10:42.160563 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:10:42.160563 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:10:42.160563 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:10:42.160563 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:10:42.160563 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application with Lock( Sep 15 12:10:42.160563 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:10:42.160563 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:10:42.160563 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:10:42.160563 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:10:42.161410 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:10:42.161410 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:10:42.161410 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:10:42.161410 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:10:42.161410 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 12:10:42.161410 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 12:10:42.161410 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 12:10:42.161410 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 12:10:42.161410 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 12:10:42.161410 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 12:10:42.161410 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 15 12:10:42.161410 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 15 12:10:42.163738 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 142/286] 10.222.0.15 () {56 vars in 1099 bytes} [Thu Sep 15 12:10:41 2022] GET /identity/v3/roles/member => generated 84 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:10:42.170270 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4413aa07-35d3-47cc-a673-6deec8ac2bcd None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:10:42.291342 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326dccd4-8832-4eda-8ff5-1485c60d8d8c None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:10:42.321132 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4413aa07-35d3-47cc-a673-6deec8ac2bcd None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:10:42.358156 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037d81c1-73f7-4712-b87f-5e464072de57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:10:42.358798 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037d81c1-73f7-4712-b87f-5e464072de57 None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:42.359080 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037d81c1-73f7-4712-b87f-5e464072de57 None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:42.359354 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037d81c1-73f7-4712-b87f-5e464072de57 None admin] PATH_INFO: `/v3/users/23e90deb1aec4e90845c059729c5f57d` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:42.361367 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-037d81c1-73f7-4712-b87f-5e464072de57 None admin] RBAC: Authorizing `identity:get_user(user_id=23e90deb1aec4e90845c059729c5f57d)` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:10:42.363497 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-037d81c1-73f7-4712-b87f-5e464072de57 None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:10:42.366902 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 145/287] 10.222.0.15 () {56 vars in 1177 bytes} [Thu Sep 15 12:10:42 2022] GET /identity/v3/users/23e90deb1aec4e90845c059729c5f57d => generated 297 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:10:42.372882 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037d81c1-73f7-4712-b87f-5e464072de57 None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:10:42.376948 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21c4d7f-b438-4f63-afc0-fbc0ce5cd967 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:10:42.377669 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21c4d7f-b438-4f63-afc0-fbc0ce5cd967 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:42.377952 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21c4d7f-b438-4f63-afc0-fbc0ce5cd967 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:42.378221 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21c4d7f-b438-4f63-afc0-fbc0ce5cd967 None admin] PATH_INFO: `/v3/roles` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:42.378886 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21c4d7f-b438-4f63-afc0-fbc0ce5cd967 None admin] RBAC: Adding query filter params (name=member) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:10:42.379184 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21c4d7f-b438-4f63-afc0-fbc0ce5cd967 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:10:42.380551 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21c4d7f-b438-4f63-afc0-fbc0ce5cd967 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:10:42.390673 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 143/288] 10.222.0.15 () {56 vars in 1101 bytes} [Thu Sep 15 12:10:42 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:10:42.398428 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21c4d7f-b438-4f63-afc0-fbc0ce5cd967 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:10:42.426219 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:10:42.461919 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:10:42.549510 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037d81c1-73f7-4712-b87f-5e464072de57 None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:10:42.595771 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c830eb7-d51e-4c7f-b8ef-6999019e6f4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:10:42.596572 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c830eb7-d51e-4c7f-b8ef-6999019e6f4f None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:42.596883 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c830eb7-d51e-4c7f-b8ef-6999019e6f4f None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:42.597479 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c830eb7-d51e-4c7f-b8ef-6999019e6f4f None admin] PATH_INFO: `/v3/projects/bd9c06e0f78e4d26902fd521a2fe013e` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:42.598814 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c830eb7-d51e-4c7f-b8ef-6999019e6f4f None admin] RBAC: Authorizing `identity:get_project(project_id=bd9c06e0f78e4d26902fd521a2fe013e)` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:10:42.600685 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c830eb7-d51e-4c7f-b8ef-6999019e6f4f None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:10:42.602658 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 146/289] 10.222.0.15 () {56 vars in 1186 bytes} [Thu Sep 15 12:10:42 2022] GET /identity/v3/projects/bd9c06e0f78e4d26902fd521a2fe013e => generated 302 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:10:42.606387 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21c4d7f-b438-4f63-afc0-fbc0ce5cd967 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:10:42.614052 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c830eb7-d51e-4c7f-b8ef-6999019e6f4f None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:10:42.666602 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79cc12b-85db-45e0-8464-10f8b09fe66a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:10:42.667247 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79cc12b-85db-45e0-8464-10f8b09fe66a None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:42.667524 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79cc12b-85db-45e0-8464-10f8b09fe66a None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:42.667791 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79cc12b-85db-45e0-8464-10f8b09fe66a None admin] PATH_INFO: `/v3/users/b8b4429c9fff4109b34e2ffafd7ba319` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:42.670840 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79cc12b-85db-45e0-8464-10f8b09fe66a None admin] RBAC: Authorizing `identity:get_user(user_id=b8b4429c9fff4109b34e2ffafd7ba319)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:10:42.672272 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79cc12b-85db-45e0-8464-10f8b09fe66a None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:10:42.674941 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 144/290] 10.222.0.15 () {56 vars in 1177 bytes} [Thu Sep 15 12:10:42 2022] GET /identity/v3/users/b8b4429c9fff4109b34e2ffafd7ba319 => generated 297 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:10:42.683181 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79cc12b-85db-45e0-8464-10f8b09fe66a None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:10:42.767265 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c830eb7-d51e-4c7f-b8ef-6999019e6f4f None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:10:42.817228 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:10:42.834481 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:10:42.853460 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85c4e21-501a-42a8-94fe-cc4f8cbd5177 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:10:42.854473 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85c4e21-501a-42a8-94fe-cc4f8cbd5177 None admin] REQUEST_METHOD: `PUT` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:42.855210 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85c4e21-501a-42a8-94fe-cc4f8cbd5177 None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:42.855712 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85c4e21-501a-42a8-94fe-cc4f8cbd5177 None admin] PATH_INFO: `/v3/projects/bd9c06e0f78e4d26902fd521a2fe013e/users/23e90deb1aec4e90845c059729c5f57d/roles/1807da0df67b49739d033f8c0e7456f0` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:42.858130 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85c4e21-501a-42a8-94fe-cc4f8cbd5177 None admin] RBAC: Authorizing `identity:create_grant(project_id=bd9c06e0f78e4d26902fd521a2fe013e, user_id=23e90deb1aec4e90845c059729c5f57d, role_id=1807da0df67b49739d033f8c0e7456f0)` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:10:42.859811 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d85c4e21-501a-42a8-94fe-cc4f8cbd5177 None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:10:42.860939 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79cc12b-85db-45e0-8464-10f8b09fe66a None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:10:42.894800 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 147/291] 10.222.0.15 () {58 vars in 1439 bytes} [Thu Sep 15 12:10:42 2022] PUT /identity/v3/projects/bd9c06e0f78e4d26902fd521a2fe013e/users/23e90deb1aec4e90845c059729c5f57d/roles/1807da0df67b49739d033f8c0e7456f0 => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:10:42.906192 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33414b7-cb60-4e3a-9df8-d9b47a83a036 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:10:42.906827 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33414b7-cb60-4e3a-9df8-d9b47a83a036 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:42.907103 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33414b7-cb60-4e3a-9df8-d9b47a83a036 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:42.907368 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33414b7-cb60-4e3a-9df8-d9b47a83a036 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:43.008493 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f33414b7-cb60-4e3a-9df8-d9b47a83a036 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:10:43.010039 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f33414b7-cb60-4e3a-9df8-d9b47a83a036 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:10:43.096869 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:10:43.124440 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 145/292] 10.222.0.15 () {58 vars in 1282 bytes} [Thu Sep 15 12:10:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 442 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 15 12:10:43.133865 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85c4e21-501a-42a8-94fe-cc4f8cbd5177 None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:10:43.276120 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85c4e21-501a-42a8-94fe-cc4f8cbd5177 None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:10:43.312695 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93785a54-a43f-436b-ac2e-f079b5790de6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:10:43.313483 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93785a54-a43f-436b-ac2e-f079b5790de6 None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:43.314040 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93785a54-a43f-436b-ac2e-f079b5790de6 None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:43.314310 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93785a54-a43f-436b-ac2e-f079b5790de6 None admin] PATH_INFO: `/v3/projects/bd9c06e0f78e4d26902fd521a2fe013e` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:43.315475 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93785a54-a43f-436b-ac2e-f079b5790de6 None admin] RBAC: Authorizing `identity:get_project(project_id=bd9c06e0f78e4d26902fd521a2fe013e)` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:10:43.317004 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93785a54-a43f-436b-ac2e-f079b5790de6 None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:10:43.318920 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 148/293] 10.222.0.15 () {56 vars in 1186 bytes} [Thu Sep 15 12:10:43 2022] GET /identity/v3/projects/bd9c06e0f78e4d26902fd521a2fe013e => generated 302 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:10:43.324835 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33414b7-cb60-4e3a-9df8-d9b47a83a036 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:10:43.454248 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33414b7-cb60-4e3a-9df8-d9b47a83a036 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:10:43.499797 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095d549b-564f-45d1-9659-90c9ece13082 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:10:43.500416 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095d549b-564f-45d1-9659-90c9ece13082 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:43.500693 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095d549b-564f-45d1-9659-90c9ece13082 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:43.500983 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095d549b-564f-45d1-9659-90c9ece13082 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:43.501703 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095d549b-564f-45d1-9659-90c9ece13082 None admin] RBAC: Adding query filter params (role.id=0679d6628a25438d90dc4bdc0401eda2, scope.project.id=bd9c06e0f78e4d26902fd521a2fe013e, user.id=b8b4429c9fff4109b34e2ffafd7ba319) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:10:43.502007 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095d549b-564f-45d1-9659-90c9ece13082 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:10:43.503447 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095d549b-564f-45d1-9659-90c9ece13082 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:10:43.509796 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 146/294] 10.222.0.15 () {56 vars in 1374 bytes} [Thu Sep 15 12:10:43 2022] GET /identity/v3/role_assignments?user.id=b8b4429c9fff4109b34e2ffafd7ba319&scope.project.id=bd9c06e0f78e4d26902fd521a2fe013e&role.id=0679d6628a25438d90dc4bdc0401eda2 => generated 608 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:10:44.401309 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ccd012-b6ba-413a-aed4-f42117975556 None None] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:44.401814 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ccd012-b6ba-413a-aed4-f42117975556 None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:44.401814 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ccd012-b6ba-413a-aed4-f42117975556 None None] PATH_INFO: `/` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:44.403362 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 149/295] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 12:10:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:10:44.408499 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d89ad5e-10da-45d4-a57d-f6e61eb96c49 None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:44.408948 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d89ad5e-10da-45d4-a57d-f6e61eb96c49 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:44.409042 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d89ad5e-10da-45d4-a57d-f6e61eb96c49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:44.429226 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-4d89ad5e-10da-45d4-a57d-f6e61eb96c49 None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:10:44.430472 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-4d89ad5e-10da-45d4-a57d-f6e61eb96c49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:10:44.472409 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:10:44.507132 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 147/296] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 12:10:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 12:10:44.556214 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7840bed2-8cbd-47b9-8971-e92dedee393b None None] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:44.556547 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7840bed2-8cbd-47b9-8971-e92dedee393b None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:44.556789 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7840bed2-8cbd-47b9-8971-e92dedee393b None None] PATH_INFO: `/` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:44.558123 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 150/297] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 12:10:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:10:44.563092 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7162e7-a863-4cc0-a84d-af7aff459f71 None None] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:44.563212 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7542d9f-f859-41b6-b4d9-5090bd0b4548 None None] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:44.563403 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7162e7-a863-4cc0-a84d-af7aff459f71 None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:44.563496 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7542d9f-f859-41b6-b4d9-5090bd0b4548 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:44.563639 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7162e7-a863-4cc0-a84d-af7aff459f71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:44.563727 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7542d9f-f859-41b6-b4d9-5090bd0b4548 None None] PATH_INFO: `/` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:44.565265 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 148/298] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 12:10:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:10:44.570902 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cb4f2b-0219-445b-9dd7-a0870d748970 None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:44.571214 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cb4f2b-0219-445b-9dd7-a0870d748970 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:44.571454 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cb4f2b-0219-445b-9dd7-a0870d748970 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:44.583929 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.auth.core [None req-2d7162e7-a863-4cc0-a84d-af7aff459f71 None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81449) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:10:44.585216 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-2d7162e7-a863-4cc0-a84d-af7aff459f71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:10:44.591283 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-76cb4f2b-0219-445b-9dd7-a0870d748970 None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:10:44.592449 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-76cb4f2b-0219-445b-9dd7-a0870d748970 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:10:44.643898 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:10:44.667573 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 149/299] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 12:10:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 15 12:10:44.676779 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 151/300] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 12:10:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 15 12:10:44.786389 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76cb4f2b-0219-445b-9dd7-a0870d748970 None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:10:44.884367 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:10:44.887929 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76cb4f2b-0219-445b-9dd7-a0870d748970 None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:10:44.927000 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b8f187-6a4c-4384-8675-0869b69add2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:10:44.927984 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b8f187-6a4c-4384-8675-0869b69add2a None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:44.929054 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b8f187-6a4c-4384-8675-0869b69add2a None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:44.929537 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b8f187-6a4c-4384-8675-0869b69add2a None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:44.940069 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b8f187-6a4c-4384-8675-0869b69add2a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 12:10:44.945178 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application [None req-f7b8f187-6a4c-4384-8675-0869b69add2a None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 15 12:10:44.945178 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:10:44.945178 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:10:44.945178 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:10:44.945178 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:10:44.945178 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:10:44.945178 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:10:44.945178 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:10:44.945178 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:10:44.945178 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:10:44.945178 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:10:44.945178 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:10:44.945178 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 12:10:44.945178 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 12:10:44.945178 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 12:10:44.945178 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 12:10:44.946066 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 12:10:44.946066 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 12:10:44.946066 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 12:10:44.946066 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 12:10:44.946066 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:10:44.946066 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:10:44.946066 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:10:44.946066 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:10:44.946066 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:10:44.946066 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:10:44.946066 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:10:44.946066 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application with Lock( Sep 15 12:10:44.946066 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:10:44.946066 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:10:44.946066 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:10:44.946066 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:10:44.946845 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:10:44.946845 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:10:44.946845 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:10:44.946845 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:10:44.946845 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 12:10:44.946845 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 12:10:44.946845 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 12:10:44.946845 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 12:10:44.946845 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 12:10:44.946845 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 12:10:44.946845 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 15 12:10:44.946845 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 15 12:10:44.947470 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7162e7-a863-4cc0-a84d-af7aff459f71 None None] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:10:44.949753 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 150/301] 10.222.0.15 () {56 vars in 1099 bytes} [Thu Sep 15 12:10:44 2022] GET /identity/v3/roles/reader => generated 84 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:10:44.958288 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b8f187-6a4c-4384-8675-0869b69add2a None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:10:45.054105 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7162e7-a863-4cc0-a84d-af7aff459f71 None None] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:10:45.072335 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b8f187-6a4c-4384-8675-0869b69add2a None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:10:45.094918 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8e121f-51db-40a9-b527-3cd54b7979f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:10:45.095666 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8e121f-51db-40a9-b527-3cd54b7979f7 None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:45.095939 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8e121f-51db-40a9-b527-3cd54b7979f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:45.096206 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8e121f-51db-40a9-b527-3cd54b7979f7 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:45.107125 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb8e121f-51db-40a9-b527-3cd54b7979f7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 12:10:45.112228 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application [None req-cb8e121f-51db-40a9-b527-3cd54b7979f7 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 15 12:10:45.112228 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:10:45.112228 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:10:45.112228 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:10:45.112228 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:10:45.112228 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:10:45.112228 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:10:45.112228 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:10:45.112228 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:10:45.112228 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:10:45.112228 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:10:45.112228 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:10:45.112228 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 12:10:45.112228 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 12:10:45.112228 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 12:10:45.112228 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 12:10:45.113136 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 12:10:45.113136 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 12:10:45.113136 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 12:10:45.113136 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 12:10:45.113136 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:10:45.113136 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:10:45.113136 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:10:45.113136 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:10:45.113136 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:10:45.113136 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:10:45.113136 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:10:45.113136 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application with Lock( Sep 15 12:10:45.113136 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:10:45.113136 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:10:45.113136 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:10:45.113136 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:10:45.113935 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:10:45.113935 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:10:45.113935 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:10:45.113935 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:10:45.113935 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 12:10:45.113935 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 12:10:45.113935 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 12:10:45.113935 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 12:10:45.113935 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 12:10:45.113935 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 12:10:45.113935 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 15 12:10:45.113935 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application Sep 15 12:10:45.114565 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452406fa-de26-447c-926c-8409256b1ef2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:10:45.114565 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452406fa-de26-447c-926c-8409256b1ef2 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:45.114565 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452406fa-de26-447c-926c-8409256b1ef2 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:45.114760 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 152/302] 10.222.0.15 () {56 vars in 1099 bytes} [Thu Sep 15 12:10:44 2022] GET /identity/v3/roles/member => generated 84 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:10:45.114760 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8e121f-51db-40a9-b527-3cd54b7979f7 None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:10:45.118637 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452406fa-de26-447c-926c-8409256b1ef2 None admin] PATH_INFO: `/v3/roles` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:45.119362 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452406fa-de26-447c-926c-8409256b1ef2 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:10:45.119678 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452406fa-de26-447c-926c-8409256b1ef2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:10:45.121147 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452406fa-de26-447c-926c-8409256b1ef2 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:10:45.129892 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 151/303] 10.222.0.15 () {56 vars in 1101 bytes} [Thu Sep 15 12:10:44 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:10:45.130679 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452406fa-de26-447c-926c-8409256b1ef2 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:10:45.158168 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:10:45.164777 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:10:45.222215 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8e121f-51db-40a9-b527-3cd54b7979f7 None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:10:45.243279 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452406fa-de26-447c-926c-8409256b1ef2 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:10:45.257715 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417237ae-1bec-4956-8f30-d9117a16f1d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:10:45.258430 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417237ae-1bec-4956-8f30-d9117a16f1d9 None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:45.258729 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417237ae-1bec-4956-8f30-d9117a16f1d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:45.259010 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417237ae-1bec-4956-8f30-d9117a16f1d9 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:45.260056 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417237ae-1bec-4956-8f30-d9117a16f1d9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:10:45.261528 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417237ae-1bec-4956-8f30-d9117a16f1d9 None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:10:45.263227 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 153/304] 10.222.0.15 () {56 vars in 1108 bytes} [Thu Sep 15 12:10:45 2022] GET /identity/v3/domains/default => generated 199 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:10:45.268209 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417237ae-1bec-4956-8f30-d9117a16f1d9 None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:10:45.278655 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2cab8c-212f-4f73-a1dc-062c637d1ee4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:10:45.279272 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2cab8c-212f-4f73-a1dc-062c637d1ee4 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:45.279555 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2cab8c-212f-4f73-a1dc-062c637d1ee4 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:45.279832 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2cab8c-212f-4f73-a1dc-062c637d1ee4 None admin] PATH_INFO: `/v3/roles` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:45.280495 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2cab8c-212f-4f73-a1dc-062c637d1ee4 None admin] RBAC: Adding query filter params (name=member) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:10:45.280861 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2cab8c-212f-4f73-a1dc-062c637d1ee4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:10:45.282222 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2cab8c-212f-4f73-a1dc-062c637d1ee4 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:10:45.291988 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 152/305] 10.222.0.15 () {56 vars in 1101 bytes} [Thu Sep 15 12:10:45 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:10:45.292675 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2cab8c-212f-4f73-a1dc-062c637d1ee4 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:10:45.388230 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417237ae-1bec-4956-8f30-d9117a16f1d9 None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:10:45.389037 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:10:45.402445 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:10:45.407931 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2cab8c-212f-4f73-a1dc-062c637d1ee4 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:10:45.426468 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150d8126-85a4-49e1-aca5-bf9a53300cd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:10:45.427106 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150d8126-85a4-49e1-aca5-bf9a53300cd2 None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:45.427381 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150d8126-85a4-49e1-aca5-bf9a53300cd2 None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:45.427651 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150d8126-85a4-49e1-aca5-bf9a53300cd2 None admin] PATH_INFO: `/v3/users/3300d8c83014462fa5da513a07518836` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:45.440676 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-150d8126-85a4-49e1-aca5-bf9a53300cd2 None admin] RBAC: Authorizing `identity:get_user(user_id=3300d8c83014462fa5da513a07518836)` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:10:45.442140 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-150d8126-85a4-49e1-aca5-bf9a53300cd2 None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:10:45.443196 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6af93e-2a6c-4d4f-82b2-d9401b97359d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:10:45.443805 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 154/306] 10.222.0.15 () {56 vars in 1177 bytes} [Thu Sep 15 12:10:45 2022] GET /identity/v3/users/3300d8c83014462fa5da513a07518836 => generated 293 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:10:45.443910 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6af93e-2a6c-4d4f-82b2-d9401b97359d None admin] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:45.444169 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6af93e-2a6c-4d4f-82b2-d9401b97359d None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:45.444449 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6af93e-2a6c-4d4f-82b2-d9401b97359d None admin] PATH_INFO: `/v3/groups` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:45.444600 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150d8126-85a4-49e1-aca5-bf9a53300cd2 None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:10:45.445195 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6af93e-2a6c-4d4f-82b2-d9401b97359d None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:10:45.446610 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6af93e-2a6c-4d4f-82b2-d9401b97359d None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:10:45.462515 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 153/307] 10.222.0.15 () {60 vars in 1134 bytes} [Thu Sep 15 12:10:45 2022] POST /identity/v3/groups => generated 229 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 12:10:45.464847 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bb6501-f7f6-4ef4-b69d-032fd915d511 None None] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:45.464975 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bb6501-f7f6-4ef4-b69d-032fd915d511 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:45.465224 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54bb6501-f7f6-4ef4-b69d-032fd915d511 None None] PATH_INFO: `/` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:45.468896 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 154/308] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 12:10:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:10:45.468896 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54bb6501-f7f6-4ef4-b69d-032fd915d511 None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:10:45.554964 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150d8126-85a4-49e1-aca5-bf9a53300cd2 None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:10:45.594155 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48650416-6096-458c-8b0e-03d00cb40919 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:10:45.594918 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48650416-6096-458c-8b0e-03d00cb40919 None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:45.595197 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48650416-6096-458c-8b0e-03d00cb40919 None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:45.595467 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48650416-6096-458c-8b0e-03d00cb40919 None admin] PATH_INFO: `/v3/users/2f30ab0c797045a5bdc70b4e0b90a87e` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:45.596451 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54bb6501-f7f6-4ef4-b69d-032fd915d511 None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:10:45.596635 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48650416-6096-458c-8b0e-03d00cb40919 None admin] RBAC: Authorizing `identity:get_user(user_id=2f30ab0c797045a5bdc70b4e0b90a87e)` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:10:45.598122 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48650416-6096-458c-8b0e-03d00cb40919 None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:10:45.599895 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 155/309] 10.222.0.15 () {56 vars in 1177 bytes} [Thu Sep 15 12:10:45 2022] GET /identity/v3/users/2f30ab0c797045a5bdc70b4e0b90a87e => generated 293 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:10:45.601008 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f848d5-e2f1-421b-b220-52e528267eaf None None] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:45.601290 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f848d5-e2f1-421b-b220-52e528267eaf None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:45.601562 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f848d5-e2f1-421b-b220-52e528267eaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:45.620549 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.auth.core [None req-24f848d5-e2f1-421b-b220-52e528267eaf None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81449) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:10:45.621813 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-24f848d5-e2f1-421b-b220-52e528267eaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:10:45.643257 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89cc556a-ee87-47bb-bc56-b1af8609cf7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:10:45.644009 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cc556a-ee87-47bb-bc56-b1af8609cf7d None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:45.644287 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cc556a-ee87-47bb-bc56-b1af8609cf7d None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:45.644553 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cc556a-ee87-47bb-bc56-b1af8609cf7d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:45.645342 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89cc556a-ee87-47bb-bc56-b1af8609cf7d None admin] RBAC: Adding query filter params (role.id=1807da0df67b49739d033f8c0e7456f0, user.id=3300d8c83014462fa5da513a07518836, scope.system=all) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:10:45.645667 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89cc556a-ee87-47bb-bc56-b1af8609cf7d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:10:45.647697 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89cc556a-ee87-47bb-bc56-b1af8609cf7d None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:10:45.653433 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 155/310] 10.222.0.15 () {56 vars in 1308 bytes} [Thu Sep 15 12:10:45 2022] GET /identity/v3/role_assignments?user.id=3300d8c83014462fa5da513a07518836&scope.system=all&role.id=1807da0df67b49739d033f8c0e7456f0 => generated 226 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:10:45.654209 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89cc556a-ee87-47bb-bc56-b1af8609cf7d None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:10:45.666851 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:10:45.669670 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:10:45.739825 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 156/311] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 12:10:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 12:10:45.781288 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89cc556a-ee87-47bb-bc56-b1af8609cf7d None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:10:45.819969 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aecb280-454a-47d9-bc02-581a45942dd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:10:45.820618 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aecb280-454a-47d9-bc02-581a45942dd2 None admin] REQUEST_METHOD: `PUT` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:45.820929 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aecb280-454a-47d9-bc02-581a45942dd2 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:45.821196 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aecb280-454a-47d9-bc02-581a45942dd2 None admin] PATH_INFO: `/v3/system/users/2f30ab0c797045a5bdc70b4e0b90a87e/roles/0679d6628a25438d90dc4bdc0401eda2` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:45.822715 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aecb280-454a-47d9-bc02-581a45942dd2 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=2f30ab0c797045a5bdc70b4e0b90a87e, role_id=0679d6628a25438d90dc4bdc0401eda2)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:10:45.824090 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aecb280-454a-47d9-bc02-581a45942dd2 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:10:45.838784 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 156/312] 10.222.0.15 () {58 vars in 1334 bytes} [Thu Sep 15 12:10:45 2022] PUT /identity/v3/system/users/2f30ab0c797045a5bdc70b4e0b90a87e/roles/0679d6628a25438d90dc4bdc0401eda2 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:10:46.012948 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f848d5-e2f1-421b-b220-52e528267eaf None None] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:10:46.117315 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f848d5-e2f1-421b-b220-52e528267eaf None None] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:10:46.152785 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825ba4c2-a04a-466e-8270-ab7c616d8c67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:10:46.153435 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825ba4c2-a04a-466e-8270-ab7c616d8c67 None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:46.153710 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825ba4c2-a04a-466e-8270-ab7c616d8c67 None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:46.153980 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825ba4c2-a04a-466e-8270-ab7c616d8c67 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:46.164824 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-825ba4c2-a04a-466e-8270-ab7c616d8c67 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 12:10:46.169675 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application [None req-825ba4c2-a04a-466e-8270-ab7c616d8c67 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 15 12:10:46.169675 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:10:46.169675 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:10:46.169675 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:10:46.169675 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:10:46.169675 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:10:46.169675 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:10:46.169675 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:10:46.169675 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:10:46.169675 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:10:46.169675 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:10:46.169675 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:10:46.169675 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 12:10:46.169675 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 12:10:46.169675 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 12:10:46.169675 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 12:10:46.170561 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 12:10:46.170561 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 12:10:46.170561 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 12:10:46.170561 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 12:10:46.170561 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:10:46.170561 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:10:46.170561 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:10:46.170561 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:10:46.170561 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:10:46.170561 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:10:46.170561 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:10:46.170561 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application with Lock( Sep 15 12:10:46.170561 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:10:46.170561 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:10:46.170561 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:10:46.170561 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:10:46.171355 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:10:46.171355 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:10:46.171355 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:10:46.171355 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:10:46.171355 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 12:10:46.171355 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 12:10:46.171355 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 12:10:46.171355 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 12:10:46.171355 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 12:10:46.171355 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 12:10:46.171355 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 15 12:10:46.171355 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application Sep 15 12:10:46.171355 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 157/313] 10.222.0.15 () {56 vars in 1099 bytes} [Thu Sep 15 12:10:46 2022] GET /identity/v3/roles/reader => generated 84 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:10:46.176740 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aecb280-454a-47d9-bc02-581a45942dd2 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:10:46.250766 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:10:46.289119 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aecb280-454a-47d9-bc02-581a45942dd2 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:10:46.324541 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ef68de-303b-4787-8bf2-8b9585a5377a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:10:46.325423 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ef68de-303b-4787-8bf2-8b9585a5377a None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:46.325888 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ef68de-303b-4787-8bf2-8b9585a5377a None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:46.326307 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ef68de-303b-4787-8bf2-8b9585a5377a None admin] PATH_INFO: `/v3/roles` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:46.327108 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ef68de-303b-4787-8bf2-8b9585a5377a None admin] RBAC: Adding query filter params (name=reader) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:10:46.327553 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ef68de-303b-4787-8bf2-8b9585a5377a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:10:46.329084 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ef68de-303b-4787-8bf2-8b9585a5377a None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:10:46.338512 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 157/314] 10.222.0.15 () {56 vars in 1101 bytes} [Thu Sep 15 12:10:46 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:10:46.344336 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825ba4c2-a04a-466e-8270-ab7c616d8c67 None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:10:46.375251 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:10:46.456925 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825ba4c2-a04a-466e-8270-ab7c616d8c67 None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:10:46.495013 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5be45b8-5b13-463f-8a21-0e9269ba08a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:10:46.496013 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5be45b8-5b13-463f-8a21-0e9269ba08a1 None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:46.496492 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5be45b8-5b13-463f-8a21-0e9269ba08a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:46.497009 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5be45b8-5b13-463f-8a21-0e9269ba08a1 None admin] PATH_INFO: `/v3/users/23e90deb1aec4e90845c059729c5f57d` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:46.498324 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5be45b8-5b13-463f-8a21-0e9269ba08a1 None admin] RBAC: Authorizing `identity:get_user(user_id=23e90deb1aec4e90845c059729c5f57d)` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:10:46.499969 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5be45b8-5b13-463f-8a21-0e9269ba08a1 None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:10:46.501760 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 158/315] 10.222.0.15 () {56 vars in 1177 bytes} [Thu Sep 15 12:10:46 2022] GET /identity/v3/users/23e90deb1aec4e90845c059729c5f57d => generated 297 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:10:46.509851 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ef68de-303b-4787-8bf2-8b9585a5377a None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:10:46.617040 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ef68de-303b-4787-8bf2-8b9585a5377a None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:10:46.651151 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3777c355-14a3-4bed-af9a-930303deedb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:10:46.651688 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3777c355-14a3-4bed-af9a-930303deedb6 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:46.651970 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3777c355-14a3-4bed-af9a-930303deedb6 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:46.652241 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3777c355-14a3-4bed-af9a-930303deedb6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:46.655011 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:10:46.723603 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3777c355-14a3-4bed-af9a-930303deedb6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:10:46.726058 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3777c355-14a3-4bed-af9a-930303deedb6 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:10:46.830544 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 158/316] 10.222.0.15 () {58 vars in 1282 bytes} [Thu Sep 15 12:10:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 15 12:10:46.837886 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5be45b8-5b13-463f-8a21-0e9269ba08a1 None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:10:46.945539 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5be45b8-5b13-463f-8a21-0e9269ba08a1 None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:10:46.949094 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:10:46.982459 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa75db44-cbeb-4ac7-ad3b-efccbc61c2a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:10:46.983102 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa75db44-cbeb-4ac7-ad3b-efccbc61c2a7 None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:46.983381 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa75db44-cbeb-4ac7-ad3b-efccbc61c2a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:46.983654 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa75db44-cbeb-4ac7-ad3b-efccbc61c2a7 None admin] PATH_INFO: `/v3/projects/bd9c06e0f78e4d26902fd521a2fe013e` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:46.984781 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa75db44-cbeb-4ac7-ad3b-efccbc61c2a7 None admin] RBAC: Authorizing `identity:get_project(project_id=bd9c06e0f78e4d26902fd521a2fe013e)` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:10:46.986236 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa75db44-cbeb-4ac7-ad3b-efccbc61c2a7 None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:10:46.987848 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 159/317] 10.222.0.15 () {56 vars in 1186 bytes} [Thu Sep 15 12:10:46 2022] GET /identity/v3/projects/bd9c06e0f78e4d26902fd521a2fe013e => generated 302 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:10:46.997834 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3777c355-14a3-4bed-af9a-930303deedb6 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:10:47.032301 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:10:47.098557 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3777c355-14a3-4bed-af9a-930303deedb6 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:10:47.133375 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f2681d-023b-4b2f-9c96-6e78a2355ac1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:10:47.133989 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f2681d-023b-4b2f-9c96-6e78a2355ac1 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:47.134267 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f2681d-023b-4b2f-9c96-6e78a2355ac1 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:47.134538 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f2681d-023b-4b2f-9c96-6e78a2355ac1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:47.135260 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f2681d-023b-4b2f-9c96-6e78a2355ac1 None admin] RBAC: Adding query filter params (role.id=1807da0df67b49739d033f8c0e7456f0, scope.project.id=bd9c06e0f78e4d26902fd521a2fe013e, user.id=23e90deb1aec4e90845c059729c5f57d) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:10:47.135570 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f2681d-023b-4b2f-9c96-6e78a2355ac1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:10:47.137019 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f2681d-023b-4b2f-9c96-6e78a2355ac1 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:10:47.143112 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 159/318] 10.222.0.15 () {56 vars in 1374 bytes} [Thu Sep 15 12:10:46 2022] GET /identity/v3/role_assignments?user.id=23e90deb1aec4e90845c059729c5f57d&scope.project.id=bd9c06e0f78e4d26902fd521a2fe013e&role.id=1807da0df67b49739d033f8c0e7456f0 => generated 608 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:10:47.704968 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64d7aa1-99cf-4188-819a-b1d28174c91a None None] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:47.705483 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64d7aa1-99cf-4188-819a-b1d28174c91a None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:47.705559 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64d7aa1-99cf-4188-819a-b1d28174c91a None None] PATH_INFO: `/` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:47.706859 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 160/319] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 12:10:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:10:47.712888 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ef21e8-a247-4cbf-ad2b-3a29a344e802 None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:47.713302 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ef21e8-a247-4cbf-ad2b-3a29a344e802 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:47.713468 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ef21e8-a247-4cbf-ad2b-3a29a344e802 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:47.732174 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-16ef21e8-a247-4cbf-ad2b-3a29a344e802 None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:10:47.733416 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-16ef21e8-a247-4cbf-ad2b-3a29a344e802 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:10:47.805739 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 160/320] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 12:10:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 12:10:47.868636 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00416ac0-0a4b-4ce9-9043-6cbc9616ad54 None None] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:47.869005 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00416ac0-0a4b-4ce9-9043-6cbc9616ad54 None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:47.869197 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00416ac0-0a4b-4ce9-9043-6cbc9616ad54 None None] PATH_INFO: `/` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:47.870655 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 161/321] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 12:10:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:10:47.875471 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdb6cec-36cc-40e7-b634-2da5d91b4128 None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:47.875836 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdb6cec-36cc-40e7-b634-2da5d91b4128 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:47.876110 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdb6cec-36cc-40e7-b634-2da5d91b4128 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:47.894986 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-2cdb6cec-36cc-40e7-b634-2da5d91b4128 None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:10:47.896148 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-2cdb6cec-36cc-40e7-b634-2da5d91b4128 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:10:47.918691 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:10:47.966619 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 161/322] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 12:10:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 12:10:48.074469 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00416ac0-0a4b-4ce9-9043-6cbc9616ad54 None None] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:10:48.183619 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00416ac0-0a4b-4ce9-9043-6cbc9616ad54 None None] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:10:48.200988 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc474fc1-b031-442d-8bf4-7102235bc1ac None None] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:48.201272 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc474fc1-b031-442d-8bf4-7102235bc1ac None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:48.201546 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc474fc1-b031-442d-8bf4-7102235bc1ac None None] PATH_INFO: `/` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:48.203049 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 162/323] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 12:10:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 15 12:10:48.207969 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc2aea3-29e4-4468-9da4-0e2231e505ee None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:48.208260 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc2aea3-29e4-4468-9da4-0e2231e505ee None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:48.208537 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc2aea3-29e4-4468-9da4-0e2231e505ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:48.219963 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1900547-f631-4d38-ba17-240ba93f57f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:10:48.220529 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1900547-f631-4d38-ba17-240ba93f57f7 None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:48.220830 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1900547-f631-4d38-ba17-240ba93f57f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:48.221101 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1900547-f631-4d38-ba17-240ba93f57f7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:48.222198 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1900547-f631-4d38-ba17-240ba93f57f7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:10:48.223653 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1900547-f631-4d38-ba17-240ba93f57f7 None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:10:48.225398 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 162/324] 10.222.0.15 () {56 vars in 1108 bytes} [Thu Sep 15 12:10:48 2022] GET /identity/v3/domains/default => generated 199 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:10:48.226927 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-efc2aea3-29e4-4468-9da4-0e2231e505ee None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:10:48.228179 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-efc2aea3-29e4-4468-9da4-0e2231e505ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:10:48.230778 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1900547-f631-4d38-ba17-240ba93f57f7 None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:10:48.261367 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:10:48.295371 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 163/325] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 12:10:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 15 12:10:48.296202 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc2aea3-29e4-4468-9da4-0e2231e505ee None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:10:48.339591 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1900547-f631-4d38-ba17-240ba93f57f7 None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:10:48.376256 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2712dccf-cbce-4058-b5b0-fc7910ae3818 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:10:48.376889 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2712dccf-cbce-4058-b5b0-fc7910ae3818 None admin] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:48.377169 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2712dccf-cbce-4058-b5b0-fc7910ae3818 None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:48.377435 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2712dccf-cbce-4058-b5b0-fc7910ae3818 None admin] PATH_INFO: `/v3/groups` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:48.378172 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2712dccf-cbce-4058-b5b0-fc7910ae3818 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:10:48.379697 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2712dccf-cbce-4058-b5b0-fc7910ae3818 None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:10:48.385502 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:10:48.398417 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 163/326] 10.222.0.15 () {60 vars in 1134 bytes} [Thu Sep 15 12:10:48 2022] POST /identity/v3/groups => generated 226 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 15 12:10:48.400789 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc2aea3-29e4-4468-9da4-0e2231e505ee None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:10:48.439557 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ebf498-eba9-424f-bf7c-972e15de9237 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:10:48.440214 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ebf498-eba9-424f-bf7c-972e15de9237 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:48.440492 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ebf498-eba9-424f-bf7c-972e15de9237 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:48.440813 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ebf498-eba9-424f-bf7c-972e15de9237 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:48.453456 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ebf498-eba9-424f-bf7c-972e15de9237 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 12:10:48.458942 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application [None req-64ebf498-eba9-424f-bf7c-972e15de9237 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 15 12:10:48.458942 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:10:48.458942 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:10:48.458942 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:10:48.458942 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:10:48.458942 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:10:48.458942 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:10:48.458942 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:10:48.458942 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:10:48.458942 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:10:48.458942 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:10:48.458942 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:10:48.458942 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 12:10:48.458942 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 12:10:48.458942 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 12:10:48.458942 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 12:10:48.459920 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 12:10:48.459920 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 12:10:48.459920 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 12:10:48.459920 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 12:10:48.459920 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:10:48.459920 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:10:48.459920 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:10:48.459920 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:10:48.459920 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:10:48.459920 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:10:48.459920 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:10:48.459920 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application with Lock( Sep 15 12:10:48.459920 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:10:48.459920 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:10:48.459920 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:10:48.459920 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:10:48.465876 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:10:48.465876 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:10:48.465876 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:10:48.465876 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:10:48.465876 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 12:10:48.465876 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 12:10:48.465876 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 12:10:48.465876 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 12:10:48.465876 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 12:10:48.465876 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 12:10:48.465876 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 15 12:10:48.465876 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 15 12:10:48.465876 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 164/327] 10.222.0.15 () {56 vars in 1099 bytes} [Thu Sep 15 12:10:48 2022] GET /identity/v3/roles/reader => generated 84 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:10:48.466666 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2712dccf-cbce-4058-b5b0-fc7910ae3818 None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:10:48.567565 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ebf498-eba9-424f-bf7c-972e15de9237 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:10:48.568491 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:10:48.581659 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2712dccf-cbce-4058-b5b0-fc7910ae3818 None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:10:48.628322 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d524ff-9343-4716-a018-096c46b2686f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:10:48.628957 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d524ff-9343-4716-a018-096c46b2686f None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:48.629237 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d524ff-9343-4716-a018-096c46b2686f None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:48.629502 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d524ff-9343-4716-a018-096c46b2686f None admin] PATH_INFO: `/v3/roles` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:48.630163 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d524ff-9343-4716-a018-096c46b2686f None admin] RBAC: Adding query filter params (name=reader) {{(pid=81449) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:10:48.630570 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d524ff-9343-4716-a018-096c46b2686f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:10:48.631856 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d524ff-9343-4716-a018-096c46b2686f None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:10:48.641394 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 164/328] 10.222.0.15 () {56 vars in 1101 bytes} [Thu Sep 15 12:10:48 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:10:48.646961 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d524ff-9343-4716-a018-096c46b2686f None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:10:48.691563 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ebf498-eba9-424f-bf7c-972e15de9237 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:10:48.735361 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edbe921b-01b3-4db7-a5a6-56b74471a34e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:10:48.736127 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbe921b-01b3-4db7-a5a6-56b74471a34e None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:48.736298 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbe921b-01b3-4db7-a5a6-56b74471a34e None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:48.736569 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbe921b-01b3-4db7-a5a6-56b74471a34e None admin] PATH_INFO: `/v3/roles/member` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:48.754497 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edbe921b-01b3-4db7-a5a6-56b74471a34e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 12:10:48.760145 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application [None req-edbe921b-01b3-4db7-a5a6-56b74471a34e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 15 12:10:48.760145 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:10:48.760145 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:10:48.760145 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:10:48.760145 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:10:48.760145 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:10:48.760145 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:10:48.760145 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:10:48.760145 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:10:48.760145 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:10:48.760145 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:10:48.760145 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:10:48.760145 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 12:10:48.760145 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 12:10:48.760145 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 12:10:48.760145 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 12:10:48.761085 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 12:10:48.761085 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 12:10:48.761085 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 12:10:48.761085 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 12:10:48.761085 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:10:48.761085 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:10:48.761085 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:10:48.761085 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:10:48.761085 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:10:48.761085 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:10:48.761085 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:10:48.761085 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application with Lock( Sep 15 12:10:48.761085 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:10:48.761085 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:10:48.761085 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:10:48.761085 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:10:48.761901 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:10:48.761901 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:10:48.761901 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:10:48.761901 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:10:48.761901 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 12:10:48.761901 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 12:10:48.761901 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 12:10:48.761901 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 12:10:48.761901 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 12:10:48.761901 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 12:10:48.761901 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 15 12:10:48.761901 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 15 12:10:48.762820 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 165/329] 10.222.0.15 () {56 vars in 1099 bytes} [Thu Sep 15 12:10:48 2022] GET /identity/v3/roles/member => generated 84 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 15 12:10:48.769560 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edbe921b-01b3-4db7-a5a6-56b74471a34e None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:10:48.776363 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d524ff-9343-4716-a018-096c46b2686f None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:10:48.782572 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:10:48.825914 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db943b6c-89b8-4ad3-b482-f6f3cd9fee8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:10:48.826795 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db943b6c-89b8-4ad3-b482-f6f3cd9fee8a None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:48.827083 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db943b6c-89b8-4ad3-b482-f6f3cd9fee8a None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:48.827361 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db943b6c-89b8-4ad3-b482-f6f3cd9fee8a None admin] PATH_INFO: `/v3/users/3300d8c83014462fa5da513a07518836` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:48.829701 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db943b6c-89b8-4ad3-b482-f6f3cd9fee8a None admin] RBAC: Authorizing `identity:get_user(user_id=3300d8c83014462fa5da513a07518836)` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:10:48.831209 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db943b6c-89b8-4ad3-b482-f6f3cd9fee8a None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:10:48.833101 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 165/330] 10.222.0.15 () {56 vars in 1177 bytes} [Thu Sep 15 12:10:48 2022] GET /identity/v3/users/3300d8c83014462fa5da513a07518836 => generated 293 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:10:48.839186 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db943b6c-89b8-4ad3-b482-f6f3cd9fee8a None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:10:48.866385 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:10:48.900627 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edbe921b-01b3-4db7-a5a6-56b74471a34e None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:10:48.937039 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6d04ba-d9de-4349-ac1a-0b2ed71d4526 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:10:48.937701 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6d04ba-d9de-4349-ac1a-0b2ed71d4526 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:48.937980 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6d04ba-d9de-4349-ac1a-0b2ed71d4526 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:48.938250 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6d04ba-d9de-4349-ac1a-0b2ed71d4526 None admin] PATH_INFO: `/v3/roles` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:48.938913 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6d04ba-d9de-4349-ac1a-0b2ed71d4526 None admin] RBAC: Adding query filter params (name=member) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:10:48.939209 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6d04ba-d9de-4349-ac1a-0b2ed71d4526 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:10:48.940587 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6d04ba-d9de-4349-ac1a-0b2ed71d4526 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:10:48.950350 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 166/331] 10.222.0.15 () {56 vars in 1101 bytes} [Thu Sep 15 12:10:48 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 12:10:48.965985 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db943b6c-89b8-4ad3-b482-f6f3cd9fee8a None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:10:48.969092 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6d04ba-d9de-4349-ac1a-0b2ed71d4526 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:10:49.008155 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27224e6-f10d-428f-9216-8e399c5ee9b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:10:49.008962 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27224e6-f10d-428f-9216-8e399c5ee9b3 None admin] REQUEST_METHOD: `PUT` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:49.009247 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27224e6-f10d-428f-9216-8e399c5ee9b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:49.009526 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27224e6-f10d-428f-9216-8e399c5ee9b3 None admin] PATH_INFO: `/v3/system/users/3300d8c83014462fa5da513a07518836/roles/1807da0df67b49739d033f8c0e7456f0` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:49.013555 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27224e6-f10d-428f-9216-8e399c5ee9b3 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=3300d8c83014462fa5da513a07518836, role_id=1807da0df67b49739d033f8c0e7456f0)` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:10:49.015057 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27224e6-f10d-428f-9216-8e399c5ee9b3 None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:10:49.028109 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 166/332] 10.222.0.15 () {58 vars in 1334 bytes} [Thu Sep 15 12:10:48 2022] PUT /identity/v3/system/users/3300d8c83014462fa5da513a07518836/roles/1807da0df67b49739d033f8c0e7456f0 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:10:49.061186 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:10:49.097382 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6d04ba-d9de-4349-ac1a-0b2ed71d4526 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:10:49.142469 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cabc008a-e15e-4478-936a-5967aedef2d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:10:49.143371 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabc008a-e15e-4478-936a-5967aedef2d6 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:49.143768 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabc008a-e15e-4478-936a-5967aedef2d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:49.144162 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cabc008a-e15e-4478-936a-5967aedef2d6 None admin] PATH_INFO: `/v3/users/2f30ab0c797045a5bdc70b4e0b90a87e` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:49.145989 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cabc008a-e15e-4478-936a-5967aedef2d6 None admin] RBAC: Authorizing `identity:get_user(user_id=2f30ab0c797045a5bdc70b4e0b90a87e)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:10:49.148032 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cabc008a-e15e-4478-936a-5967aedef2d6 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:10:49.150131 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 167/333] 10.222.0.15 () {56 vars in 1177 bytes} [Thu Sep 15 12:10:48 2022] GET /identity/v3/users/2f30ab0c797045a5bdc70b4e0b90a87e => generated 293 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:10:49.156045 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27224e6-f10d-428f-9216-8e399c5ee9b3 None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:10:49.251940 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:10:49.272686 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27224e6-f10d-428f-9216-8e399c5ee9b3 None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:10:49.315392 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7fd04f-1e4f-447a-bc2b-8539f879f58b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:10:49.316009 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7fd04f-1e4f-447a-bc2b-8539f879f58b None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:49.316285 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7fd04f-1e4f-447a-bc2b-8539f879f58b None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:49.316555 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7fd04f-1e4f-447a-bc2b-8539f879f58b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:49.317307 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7fd04f-1e4f-447a-bc2b-8539f879f58b None admin] RBAC: Adding query filter params (role.id=0679d6628a25438d90dc4bdc0401eda2, user.id=2f30ab0c797045a5bdc70b4e0b90a87e, scope.system=all) {{(pid=81449) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:10:49.317657 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7fd04f-1e4f-447a-bc2b-8539f879f58b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:10:49.319226 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7fd04f-1e4f-447a-bc2b-8539f879f58b None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:10:49.334152 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 167/334] 10.222.0.15 () {56 vars in 1308 bytes} [Thu Sep 15 12:10:49 2022] GET /identity/v3/role_assignments?user.id=2f30ab0c797045a5bdc70b4e0b90a87e&scope.system=all&role.id=0679d6628a25438d90dc4bdc0401eda2 => generated 510 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:10:51.003485 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9f84c7-0eaf-48cb-8d0b-35235422218d None None] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:51.004427 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9f84c7-0eaf-48cb-8d0b-35235422218d None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:51.004928 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9f84c7-0eaf-48cb-8d0b-35235422218d None None] PATH_INFO: `/` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:51.006988 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 168/335] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 12:10:51 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:10:51.026980 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21812af-9807-41f1-ad13-66694b244ed5 None None] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:51.027454 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21812af-9807-41f1-ad13-66694b244ed5 None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:51.027549 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21812af-9807-41f1-ad13-66694b244ed5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:51.054345 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.auth.core [None req-c21812af-9807-41f1-ad13-66694b244ed5 None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81449) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:10:51.055536 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-c21812af-9807-41f1-ad13-66694b244ed5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:10:51.145948 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 168/336] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 12:10:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 12:10:51.282313 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b0253b-2584-4149-87d9-330e36be8ffb None None] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:51.282498 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b0253b-2584-4149-87d9-330e36be8ffb None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:51.282783 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b0253b-2584-4149-87d9-330e36be8ffb None None] PATH_INFO: `/` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:51.284406 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 169/337] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 12:10:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:10:51.289395 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecc0422-e82c-4dea-8149-e147688eee74 None None] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:51.289678 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecc0422-e82c-4dea-8149-e147688eee74 None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:51.289947 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecc0422-e82c-4dea-8149-e147688eee74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:51.312960 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.auth.core [None req-fecc0422-e82c-4dea-8149-e147688eee74 None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81449) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:10:51.314145 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-fecc0422-e82c-4dea-8149-e147688eee74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:10:51.394694 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 169/338] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 12:10:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 12:10:51.644067 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b0253b-2584-4149-87d9-330e36be8ffb None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:10:51.691044 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63ff81c-0aba-40a6-b466-1f57d72ef71a None None] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:51.691044 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63ff81c-0aba-40a6-b466-1f57d72ef71a None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:51.691326 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63ff81c-0aba-40a6-b466-1f57d72ef71a None None] PATH_INFO: `/` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:51.692721 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 170/339] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 12:10:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 15 12:10:51.698852 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341e69e5-e207-4e48-be7e-12b1731cbadf None None] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:51.699136 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341e69e5-e207-4e48-be7e-12b1731cbadf None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:51.699407 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341e69e5-e207-4e48-be7e-12b1731cbadf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:51.710411 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:10:51.726157 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.auth.core [None req-341e69e5-e207-4e48-be7e-12b1731cbadf None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81449) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:10:51.727356 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-341e69e5-e207-4e48-be7e-12b1731cbadf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:10:51.774737 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b0253b-2584-4149-87d9-330e36be8ffb None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:10:51.830388 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfea9b6e-d860-4683-9100-bf2e53962b66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:10:51.831048 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfea9b6e-d860-4683-9100-bf2e53962b66 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:51.831323 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfea9b6e-d860-4683-9100-bf2e53962b66 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:51.831589 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfea9b6e-d860-4683-9100-bf2e53962b66 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:51.848301 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 171/340] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 12:10:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 15 12:10:51.854435 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:10:51.859400 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfea9b6e-d860-4683-9100-bf2e53962b66 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 12:10:51.864219 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application [None req-dfea9b6e-d860-4683-9100-bf2e53962b66 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 15 12:10:51.864219 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:10:51.864219 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:10:51.864219 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:10:51.864219 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:10:51.864219 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:10:51.864219 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:10:51.864219 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:10:51.864219 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:10:51.864219 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:10:51.864219 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:10:51.864219 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:10:51.864219 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 12:10:51.864219 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 12:10:51.864219 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 12:10:51.864219 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 12:10:51.865196 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 12:10:51.865196 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 12:10:51.865196 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 12:10:51.865196 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 12:10:51.865196 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:10:51.865196 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:10:51.865196 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:10:51.865196 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:10:51.865196 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:10:51.865196 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:10:51.865196 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:10:51.865196 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application with Lock( Sep 15 12:10:51.865196 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:10:51.865196 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:10:51.865196 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:10:51.865196 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:10:51.866045 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:10:51.866045 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:10:51.866045 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:10:51.866045 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:10:51.866045 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 12:10:51.866045 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 12:10:51.866045 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 12:10:51.866045 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 12:10:51.866045 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 12:10:51.866045 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 12:10:51.866045 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 15 12:10:51.866045 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 15 12:10:51.869106 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 170/341] 10.222.0.15 () {56 vars in 1099 bytes} [Thu Sep 15 12:10:51 2022] GET /identity/v3/roles/member => generated 84 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:10:51.871444 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341e69e5-e207-4e48-be7e-12b1731cbadf None None] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:10:51.993580 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341e69e5-e207-4e48-be7e-12b1731cbadf None None] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:10:52.004297 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfea9b6e-d860-4683-9100-bf2e53962b66 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:10:52.027473 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:10:52.036881 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4adbd6-581d-4a4b-b3fa-28e522f0b8fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:10:52.036881 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4adbd6-581d-4a4b-b3fa-28e522f0b8fb None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:52.036881 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4adbd6-581d-4a4b-b3fa-28e522f0b8fb None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:52.036881 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4adbd6-581d-4a4b-b3fa-28e522f0b8fb None admin] PATH_INFO: `/v3/roles` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:52.037257 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c4adbd6-581d-4a4b-b3fa-28e522f0b8fb None admin] RBAC: Adding query filter params (name=member) {{(pid=81449) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:10:52.037257 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c4adbd6-581d-4a4b-b3fa-28e522f0b8fb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:10:52.037257 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c4adbd6-581d-4a4b-b3fa-28e522f0b8fb None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:10:52.045815 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 172/342] 10.222.0.15 () {56 vars in 1101 bytes} [Thu Sep 15 12:10:51 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:10:52.047039 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c931f0a-b500-493e-bea3-64126d62d6b5 None None] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:52.047494 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c931f0a-b500-493e-bea3-64126d62d6b5 None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:52.047909 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c931f0a-b500-493e-bea3-64126d62d6b5 None None] PATH_INFO: `/` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:52.049394 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 173/343] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 12:10:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:10:52.058619 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c931f0a-b500-493e-bea3-64126d62d6b5 None None] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:10:52.133248 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfea9b6e-d860-4683-9100-bf2e53962b66 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:10:52.168809 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6ceb50-867a-4310-af73-3bc622c24e57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:10:52.169693 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6ceb50-867a-4310-af73-3bc622c24e57 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:52.170139 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6ceb50-867a-4310-af73-3bc622c24e57 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:52.170548 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6ceb50-867a-4310-af73-3bc622c24e57 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:52.183199 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6ceb50-867a-4310-af73-3bc622c24e57 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 12:10:52.188922 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application [None req-7d6ceb50-867a-4310-af73-3bc622c24e57 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 15 12:10:52.188922 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:10:52.188922 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:10:52.188922 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:10:52.188922 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:10:52.188922 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:10:52.188922 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:10:52.188922 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:10:52.188922 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:10:52.188922 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:10:52.188922 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:10:52.188922 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:10:52.188922 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 12:10:52.188922 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 12:10:52.188922 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 12:10:52.188922 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 12:10:52.189966 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 12:10:52.189966 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 12:10:52.189966 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 12:10:52.189966 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 12:10:52.189966 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:10:52.189966 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:10:52.189966 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:10:52.189966 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:10:52.189966 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:10:52.189966 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:10:52.189966 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:10:52.189966 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application with Lock( Sep 15 12:10:52.189966 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:10:52.189966 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:10:52.189966 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:10:52.189966 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:10:52.190926 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:10:52.190926 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:10:52.190926 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:10:52.190926 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:10:52.190926 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 12:10:52.190926 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 12:10:52.190926 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 12:10:52.190926 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 12:10:52.190926 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 12:10:52.190926 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 12:10:52.190926 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 15 12:10:52.190926 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 15 12:10:52.193133 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 171/344] 10.222.0.15 () {56 vars in 1114 bytes} [Thu Sep 15 12:10:52 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 15 12:10:52.195393 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4cf5e7-a478-4e4f-b5ff-a642329527d2 None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:52.195818 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4cf5e7-a478-4e4f-b5ff-a642329527d2 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:52.196227 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4cf5e7-a478-4e4f-b5ff-a642329527d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:52.221118 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-5f4cf5e7-a478-4e4f-b5ff-a642329527d2 None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:10:52.222696 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-5f4cf5e7-a478-4e4f-b5ff-a642329527d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:10:52.264109 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c931f0a-b500-493e-bea3-64126d62d6b5 None None] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:10:52.289132 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:10:52.309117 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 172/345] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 12:10:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 12:10:52.310301 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4cf5e7-a478-4e4f-b5ff-a642329527d2 None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:10:52.313250 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa99eec6-8181-4520-96f4-c293f54e264b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:10:52.314971 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa99eec6-8181-4520-96f4-c293f54e264b None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:52.315399 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa99eec6-8181-4520-96f4-c293f54e264b None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:52.315810 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa99eec6-8181-4520-96f4-c293f54e264b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:52.400790 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:10:52.410391 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa99eec6-8181-4520-96f4-c293f54e264b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:10:52.412869 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa99eec6-8181-4520-96f4-c293f54e264b None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:10:52.440546 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4cf5e7-a478-4e4f-b5ff-a642329527d2 None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:10:52.493799 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3125d219-840d-480d-a8cf-38f2a24f12da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:10:52.494650 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3125d219-840d-480d-a8cf-38f2a24f12da None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:52.495067 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3125d219-840d-480d-a8cf-38f2a24f12da None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:52.495494 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3125d219-840d-480d-a8cf-38f2a24f12da None admin] PATH_INFO: `/v3/roles` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:52.496290 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3125d219-840d-480d-a8cf-38f2a24f12da None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:10:52.496728 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3125d219-840d-480d-a8cf-38f2a24f12da None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:10:52.498261 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3125d219-840d-480d-a8cf-38f2a24f12da None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:10:52.506753 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 173/346] 10.222.0.15 () {56 vars in 1111 bytes} [Thu Sep 15 12:10:52 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:10:52.507680 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3125d219-840d-480d-a8cf-38f2a24f12da None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:10:52.546452 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 174/347] 10.222.0.15 () {58 vars in 1282 bytes} [Thu Sep 15 12:10:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 488 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 15 12:10:52.547491 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa99eec6-8181-4520-96f4-c293f54e264b None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:10:52.572867 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:10:52.636070 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3125d219-840d-480d-a8cf-38f2a24f12da None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:10:52.661842 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:10:52.671002 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa99eec6-8181-4520-96f4-c293f54e264b None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:10:52.678120 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf03e50-6e1c-44c7-879f-83c9f5f43faa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:10:52.678771 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf03e50-6e1c-44c7-879f-83c9f5f43faa None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:52.679053 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf03e50-6e1c-44c7-879f-83c9f5f43faa None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:52.679327 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf03e50-6e1c-44c7-879f-83c9f5f43faa None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:52.692627 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf03e50-6e1c-44c7-879f-83c9f5f43faa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 12:10:52.699959 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application [None req-adf03e50-6e1c-44c7-879f-83c9f5f43faa None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Sep 15 12:10:52.699959 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:10:52.699959 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:10:52.699959 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:10:52.699959 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:10:52.699959 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:10:52.699959 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:10:52.699959 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:10:52.699959 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:10:52.699959 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:10:52.699959 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:10:52.699959 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:10:52.699959 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 12:10:52.699959 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 12:10:52.699959 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 12:10:52.699959 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 12:10:52.701111 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 12:10:52.701111 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 12:10:52.701111 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 12:10:52.701111 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 12:10:52.701111 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:10:52.701111 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:10:52.701111 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:10:52.701111 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:10:52.701111 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:10:52.701111 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:10:52.701111 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:10:52.701111 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application with Lock( Sep 15 12:10:52.701111 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:10:52.701111 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:10:52.701111 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:10:52.701111 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:10:52.701922 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:10:52.701922 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:10:52.701922 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:10:52.701922 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:10:52.701922 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 12:10:52.701922 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 12:10:52.701922 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 12:10:52.701922 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 12:10:52.701922 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 12:10:52.701922 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 12:10:52.701922 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Sep 15 12:10:52.701922 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 15 12:10:52.703492 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 174/348] 10.222.0.15 () {56 vars in 1099 bytes} [Thu Sep 15 12:10:52 2022] GET /identity/v3/roles/reader => generated 84 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:10:52.703492 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf03e50-6e1c-44c7-879f-83c9f5f43faa None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:10:52.713991 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ead7af1-9ce8-46c6-a50b-b5cf73753e4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:10:52.714900 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ead7af1-9ce8-46c6-a50b-b5cf73753e4c None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:52.715354 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ead7af1-9ce8-46c6-a50b-b5cf73753e4c None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:52.715913 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ead7af1-9ce8-46c6-a50b-b5cf73753e4c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:52.805908 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ead7af1-9ce8-46c6-a50b-b5cf73753e4c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:10:52.807956 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ead7af1-9ce8-46c6-a50b-b5cf73753e4c None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:10:52.833900 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf03e50-6e1c-44c7-879f-83c9f5f43faa None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:10:52.847148 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:10:52.879043 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0056b6-759f-43f3-9075-5add2bfc5f57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:10:52.879700 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0056b6-759f-43f3-9075-5add2bfc5f57 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:52.879983 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0056b6-759f-43f3-9075-5add2bfc5f57 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:52.880255 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0056b6-759f-43f3-9075-5add2bfc5f57 None admin] PATH_INFO: `/v3/projects/10a21ffb0b8d4800a805c76f171c40c1` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:52.881526 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0056b6-759f-43f3-9075-5add2bfc5f57 None admin] RBAC: Authorizing `identity:get_project(project_id=10a21ffb0b8d4800a805c76f171c40c1)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:10:52.885116 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0056b6-759f-43f3-9075-5add2bfc5f57 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:10:52.885565 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 175/349] 10.222.0.15 () {56 vars in 1186 bytes} [Thu Sep 15 12:10:52 2022] GET /identity/v3/projects/10a21ffb0b8d4800a805c76f171c40c1 => generated 298 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:10:52.886195 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0056b6-759f-43f3-9075-5add2bfc5f57 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:10:52.937538 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:10:52.946620 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 175/350] 10.222.0.15 () {58 vars in 1282 bytes} [Thu Sep 15 12:10:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 15 12:10:52.948806 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ead7af1-9ce8-46c6-a50b-b5cf73753e4c None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:10:53.031455 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0056b6-759f-43f3-9075-5add2bfc5f57 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:10:53.073192 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf79deb2-42f1-43db-bc84-da827e713c0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:10:53.073822 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf79deb2-42f1-43db-bc84-da827e713c0d None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:53.074108 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf79deb2-42f1-43db-bc84-da827e713c0d None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:53.074421 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf79deb2-42f1-43db-bc84-da827e713c0d None admin] PATH_INFO: `/v3/roles` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:53.075082 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf79deb2-42f1-43db-bc84-da827e713c0d None admin] RBAC: Adding query filter params (name=reader) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:10:53.075441 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf79deb2-42f1-43db-bc84-da827e713c0d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:10:53.076840 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf79deb2-42f1-43db-bc84-da827e713c0d None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:10:53.085953 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 176/351] 10.222.0.15 () {56 vars in 1101 bytes} [Thu Sep 15 12:10:52 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:10:53.086580 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf79deb2-42f1-43db-bc84-da827e713c0d None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:10:53.086828 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ead7af1-9ce8-46c6-a50b-b5cf73753e4c None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:10:53.132211 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9088b159-e798-4238-8b88-d5f2245588ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:10:53.132891 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9088b159-e798-4238-8b88-d5f2245588ed None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:53.133174 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9088b159-e798-4238-8b88-d5f2245588ed None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:53.133444 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9088b159-e798-4238-8b88-d5f2245588ed None admin] PATH_INFO: `/v3/roles/member` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:53.136714 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:10:53.149120 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9088b159-e798-4238-8b88-d5f2245588ed None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 12:10:53.156336 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application [None req-9088b159-e798-4238-8b88-d5f2245588ed None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 15 12:10:53.156336 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:10:53.156336 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:10:53.156336 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:10:53.156336 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:10:53.156336 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:10:53.156336 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:10:53.156336 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:10:53.156336 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:10:53.156336 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:10:53.156336 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:10:53.156336 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:10:53.156336 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 12:10:53.156336 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 12:10:53.156336 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 12:10:53.156336 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 12:10:53.157266 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 12:10:53.157266 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 12:10:53.157266 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 12:10:53.157266 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 12:10:53.157266 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:10:53.157266 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:10:53.157266 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:10:53.157266 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:10:53.157266 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:10:53.157266 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:10:53.157266 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:10:53.157266 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application with Lock( Sep 15 12:10:53.157266 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:10:53.157266 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:10:53.157266 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:10:53.157266 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:10:53.158060 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:10:53.158060 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:10:53.158060 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:10:53.158060 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:10:53.158060 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 12:10:53.158060 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 12:10:53.158060 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 12:10:53.158060 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 12:10:53.158060 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 12:10:53.158060 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 12:10:53.158060 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 15 12:10:53.158060 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application Sep 15 12:10:53.159164 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 176/352] 10.222.0.15 () {56 vars in 1099 bytes} [Thu Sep 15 12:10:52 2022] GET /identity/v3/roles/member => generated 84 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:10:53.164299 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9088b159-e798-4238-8b88-d5f2245588ed None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:10:53.217346 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf79deb2-42f1-43db-bc84-da827e713c0d None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:10:53.226354 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:10:53.262116 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6696d8-377d-4e2f-b7ac-4817bbcad77a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:10:53.263059 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6696d8-377d-4e2f-b7ac-4817bbcad77a None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:53.263342 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6696d8-377d-4e2f-b7ac-4817bbcad77a None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:53.263615 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6696d8-377d-4e2f-b7ac-4817bbcad77a None admin] PATH_INFO: `/v3/groups/a53b135c5f244755bca9b0669c51acd8` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:53.271290 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6696d8-377d-4e2f-b7ac-4817bbcad77a None admin] RBAC: Authorizing `identity:get_group(group_id=a53b135c5f244755bca9b0669c51acd8)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:10:53.273465 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6696d8-377d-4e2f-b7ac-4817bbcad77a None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:10:53.275584 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 177/353] 10.222.0.15 () {56 vars in 1180 bytes} [Thu Sep 15 12:10:53 2022] GET /identity/v3/groups/a53b135c5f244755bca9b0669c51acd8 => generated 226 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:10:53.276635 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6696d8-377d-4e2f-b7ac-4817bbcad77a None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:10:53.284589 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9088b159-e798-4238-8b88-d5f2245588ed None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:10:53.331251 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c7104f-23f7-4cc9-8ac6-fb1b84fa9b58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:10:53.331939 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c7104f-23f7-4cc9-8ac6-fb1b84fa9b58 None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:53.332224 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c7104f-23f7-4cc9-8ac6-fb1b84fa9b58 None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:53.332505 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c7104f-23f7-4cc9-8ac6-fb1b84fa9b58 None admin] PATH_INFO: `/v3/projects/10a21ffb0b8d4800a805c76f171c40c1` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:53.333881 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c7104f-23f7-4cc9-8ac6-fb1b84fa9b58 None admin] RBAC: Authorizing `identity:get_project(project_id=10a21ffb0b8d4800a805c76f171c40c1)` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:10:53.335485 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c7104f-23f7-4cc9-8ac6-fb1b84fa9b58 None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:10:53.337678 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 177/354] 10.222.0.15 () {56 vars in 1186 bytes} [Thu Sep 15 12:10:53 2022] GET /identity/v3/projects/10a21ffb0b8d4800a805c76f171c40c1 => generated 298 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:10:53.338424 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c7104f-23f7-4cc9-8ac6-fb1b84fa9b58 None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:10:53.395550 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6696d8-377d-4e2f-b7ac-4817bbcad77a None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:10:53.398659 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:10:53.436914 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4213d03d-84c2-4478-bf6f-954fad48e29b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:10:53.437574 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4213d03d-84c2-4478-bf6f-954fad48e29b None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:53.437844 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4213d03d-84c2-4478-bf6f-954fad48e29b None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:53.438111 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4213d03d-84c2-4478-bf6f-954fad48e29b None admin] PATH_INFO: `/v3/users/3300d8c83014462fa5da513a07518836` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:53.440382 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4213d03d-84c2-4478-bf6f-954fad48e29b None admin] RBAC: Authorizing `identity:get_user(user_id=3300d8c83014462fa5da513a07518836)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:10:53.441859 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4213d03d-84c2-4478-bf6f-954fad48e29b None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:10:53.443751 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 178/355] 10.222.0.15 () {56 vars in 1177 bytes} [Thu Sep 15 12:10:53 2022] GET /identity/v3/users/3300d8c83014462fa5da513a07518836 => generated 293 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:10:53.444529 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4213d03d-84c2-4478-bf6f-954fad48e29b None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:10:53.452036 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c7104f-23f7-4cc9-8ac6-fb1b84fa9b58 None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:10:53.477864 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:10:53.491655 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e65bcf-bc4d-44c0-b635-0d24bd7a0f9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:10:53.492502 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e65bcf-bc4d-44c0-b635-0d24bd7a0f9e None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:53.492816 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e65bcf-bc4d-44c0-b635-0d24bd7a0f9e None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:53.493099 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e65bcf-bc4d-44c0-b635-0d24bd7a0f9e None admin] PATH_INFO: `/v3/roles` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:53.493773 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e65bcf-bc4d-44c0-b635-0d24bd7a0f9e None admin] RBAC: Adding query filter params (name=member) {{(pid=81449) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:10:53.494078 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e65bcf-bc4d-44c0-b635-0d24bd7a0f9e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:10:53.495453 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e65bcf-bc4d-44c0-b635-0d24bd7a0f9e None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:10:53.505126 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 178/356] 10.222.0.15 () {56 vars in 1101 bytes} [Thu Sep 15 12:10:53 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:10:53.505868 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e65bcf-bc4d-44c0-b635-0d24bd7a0f9e None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:10:53.569399 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4213d03d-84c2-4478-bf6f-954fad48e29b None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:10:53.613119 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56506c1-f02f-4cbf-ba76-b9176245ad48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:10:53.613952 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56506c1-f02f-4cbf-ba76-b9176245ad48 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:53.614371 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56506c1-f02f-4cbf-ba76-b9176245ad48 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:53.614879 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56506c1-f02f-4cbf-ba76-b9176245ad48 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:53.615913 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56506c1-f02f-4cbf-ba76-b9176245ad48 None admin] RBAC: Adding query filter params (group.id=a53b135c5f244755bca9b0669c51acd8, role.id=0679d6628a25438d90dc4bdc0401eda2, scope.project.id=10a21ffb0b8d4800a805c76f171c40c1) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:10:53.616361 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56506c1-f02f-4cbf-ba76-b9176245ad48 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:10:53.617986 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56506c1-f02f-4cbf-ba76-b9176245ad48 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:10:53.624478 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 179/357] 10.222.0.15 () {56 vars in 1376 bytes} [Thu Sep 15 12:10:53 2022] GET /identity/v3/role_assignments?group.id=a53b135c5f244755bca9b0669c51acd8&scope.project.id=10a21ffb0b8d4800a805c76f171c40c1&role.id=0679d6628a25438d90dc4bdc0401eda2 => generated 260 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:10:53.631404 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ce1cf9-2475-4710-8e4e-edf58344900d None None] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:53.631404 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ce1cf9-2475-4710-8e4e-edf58344900d None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:53.631670 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ce1cf9-2475-4710-8e4e-edf58344900d None None] PATH_INFO: `/` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:53.634157 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e65bcf-bc4d-44c0-b635-0d24bd7a0f9e None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:10:53.634314 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 180/358] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 12:10:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:10:53.634314 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ce1cf9-2475-4710-8e4e-edf58344900d None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:10:53.690318 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e744654a-563a-43b0-bd6f-aa38786775be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:10:53.691168 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e744654a-563a-43b0-bd6f-aa38786775be None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:53.691463 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e744654a-563a-43b0-bd6f-aa38786775be None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:53.691745 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e744654a-563a-43b0-bd6f-aa38786775be None admin] PATH_INFO: `/v3/groups/a53b135c5f244755bca9b0669c51acd8` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:53.693176 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e744654a-563a-43b0-bd6f-aa38786775be None admin] RBAC: Authorizing `identity:get_group(group_id=a53b135c5f244755bca9b0669c51acd8)` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:10:53.693581 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:10:53.695692 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e744654a-563a-43b0-bd6f-aa38786775be None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:10:53.698468 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 179/359] 10.222.0.15 () {56 vars in 1180 bytes} [Thu Sep 15 12:10:53 2022] GET /identity/v3/groups/a53b135c5f244755bca9b0669c51acd8 => generated 226 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:10:53.699347 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e744654a-563a-43b0-bd6f-aa38786775be None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:10:53.780259 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:10:53.795819 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ce1cf9-2475-4710-8e4e-edf58344900d None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:10:53.833673 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e744654a-563a-43b0-bd6f-aa38786775be None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:10:53.836843 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442f8a3e-aa09-4350-92e7-a0d80f828ac0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:10:53.837363 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442f8a3e-aa09-4350-92e7-a0d80f828ac0 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:53.837661 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442f8a3e-aa09-4350-92e7-a0d80f828ac0 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:53.837969 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442f8a3e-aa09-4350-92e7-a0d80f828ac0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:53.838757 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442f8a3e-aa09-4350-92e7-a0d80f828ac0 None admin] RBAC: Adding query filter params (role.id=1807da0df67b49739d033f8c0e7456f0, user.id=3300d8c83014462fa5da513a07518836, scope.system=all) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:10:53.839072 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442f8a3e-aa09-4350-92e7-a0d80f828ac0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:10:53.840573 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442f8a3e-aa09-4350-92e7-a0d80f828ac0 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:10:53.857812 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 181/360] 10.222.0.15 () {56 vars in 1308 bytes} [Thu Sep 15 12:10:53 2022] GET /identity/v3/role_assignments?user.id=3300d8c83014462fa5da513a07518836&scope.system=all&role.id=1807da0df67b49739d033f8c0e7456f0 => generated 510 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:10:53.859261 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1071fe-e988-4fc6-9496-bc08c36c6b8d None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:53.859553 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1071fe-e988-4fc6-9496-bc08c36c6b8d None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:53.859829 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1071fe-e988-4fc6-9496-bc08c36c6b8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:53.872055 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0636bc-a341-4349-afeb-315a11777984 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:10:53.872613 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0636bc-a341-4349-afeb-315a11777984 None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:53.873001 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0636bc-a341-4349-afeb-315a11777984 None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:53.873279 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0636bc-a341-4349-afeb-315a11777984 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:53.881525 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-bc1071fe-e988-4fc6-9496-bc08c36c6b8d None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:10:53.882813 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-bc1071fe-e988-4fc6-9496-bc08c36c6b8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:10:53.951317 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0636bc-a341-4349-afeb-315a11777984 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:10:53.954116 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0636bc-a341-4349-afeb-315a11777984 None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:10:53.967272 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 182/361] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 12:10:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 12:10:53.968290 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1071fe-e988-4fc6-9496-bc08c36c6b8d None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:10:53.998665 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:10:54.017152 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:10:54.065964 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 180/362] 10.222.0.15 () {58 vars in 1282 bytes} [Thu Sep 15 12:10:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 15 12:10:54.067260 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed403c81-bec6-4f39-94fa-555cc1c0643c None None] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:54.067713 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed403c81-bec6-4f39-94fa-555cc1c0643c None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:54.068130 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed403c81-bec6-4f39-94fa-555cc1c0643c None None] PATH_INFO: `/` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:54.073932 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 181/363] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 12:10:54 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:10:54.076430 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed403c81-bec6-4f39-94fa-555cc1c0643c None None] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:10:54.085989 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1071fe-e988-4fc6-9496-bc08c36c6b8d None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:10:54.126951 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b80f73-abdd-4f52-9a5e-3a48e139b388 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:10:54.127545 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b80f73-abdd-4f52-9a5e-3a48e139b388 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:54.127837 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b80f73-abdd-4f52-9a5e-3a48e139b388 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:54.128097 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b80f73-abdd-4f52-9a5e-3a48e139b388 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:54.129027 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b80f73-abdd-4f52-9a5e-3a48e139b388 None admin] RBAC: Adding query filter params (group.id=a53b135c5f244755bca9b0669c51acd8, role.id=1081b93ddfd547649ba8582f7a6e719e, scope.project.id=10a21ffb0b8d4800a805c76f171c40c1) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:10:54.129481 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b80f73-abdd-4f52-9a5e-3a48e139b388 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:10:54.131097 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b80f73-abdd-4f52-9a5e-3a48e139b388 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:10:54.140978 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 183/364] 10.222.0.15 () {56 vars in 1376 bytes} [Thu Sep 15 12:10:53 2022] GET /identity/v3/role_assignments?group.id=a53b135c5f244755bca9b0669c51acd8&scope.project.id=10a21ffb0b8d4800a805c76f171c40c1&role.id=1081b93ddfd547649ba8582f7a6e719e => generated 260 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:10:54.142654 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b394d5a-4ca4-4073-bcf8-fc2c5347a6ed None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:54.143131 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b394d5a-4ca4-4073-bcf8-fc2c5347a6ed None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:54.143551 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b394d5a-4ca4-4073-bcf8-fc2c5347a6ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:54.166475 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-9b394d5a-4ca4-4073-bcf8-fc2c5347a6ed None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:10:54.167981 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-9b394d5a-4ca4-4073-bcf8-fc2c5347a6ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:10:54.218936 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed403c81-bec6-4f39-94fa-555cc1c0643c None None] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:10:54.249513 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 184/365] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 12:10:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 12:10:54.269145 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6235968-3f70-4e00-bb00-a0db5dc97e96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:10:54.270154 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6235968-3f70-4e00-bb00-a0db5dc97e96 None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:54.270600 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6235968-3f70-4e00-bb00-a0db5dc97e96 None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:54.271031 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6235968-3f70-4e00-bb00-a0db5dc97e96 None admin] PATH_INFO: `/v3/projects/bd9c06e0f78e4d26902fd521a2fe013e` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:54.272368 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6235968-3f70-4e00-bb00-a0db5dc97e96 None admin] RBAC: Authorizing `identity:get_project(project_id=bd9c06e0f78e4d26902fd521a2fe013e)` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:10:54.274096 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6235968-3f70-4e00-bb00-a0db5dc97e96 None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:10:54.276353 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 182/366] 10.222.0.15 () {56 vars in 1186 bytes} [Thu Sep 15 12:10:54 2022] GET /identity/v3/projects/bd9c06e0f78e4d26902fd521a2fe013e => generated 302 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:10:54.281702 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b394d5a-4ca4-4073-bcf8-fc2c5347a6ed None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:10:54.303798 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:10:54.414120 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b394d5a-4ca4-4073-bcf8-fc2c5347a6ed None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:10:54.450091 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0307dfa9-f5fe-450a-9ffd-4aef20fc1132 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:10:54.450984 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0307dfa9-f5fe-450a-9ffd-4aef20fc1132 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:54.451412 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0307dfa9-f5fe-450a-9ffd-4aef20fc1132 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:54.451854 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0307dfa9-f5fe-450a-9ffd-4aef20fc1132 None admin] PATH_INFO: `/v3/groups/a53b135c5f244755bca9b0669c51acd8` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:54.453207 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0307dfa9-f5fe-450a-9ffd-4aef20fc1132 None admin] RBAC: Authorizing `identity:get_group(group_id=a53b135c5f244755bca9b0669c51acd8)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:10:54.454898 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0307dfa9-f5fe-450a-9ffd-4aef20fc1132 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:10:54.456628 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 185/367] 10.222.0.15 () {56 vars in 1180 bytes} [Thu Sep 15 12:10:54 2022] GET /identity/v3/groups/a53b135c5f244755bca9b0669c51acd8 => generated 226 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:10:54.463190 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6235968-3f70-4e00-bb00-a0db5dc97e96 None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:10:54.493020 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:10:54.504418 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0307dfa9-f5fe-450a-9ffd-4aef20fc1132 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:10:54.606261 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6235968-3f70-4e00-bb00-a0db5dc97e96 None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:10:54.621156 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:10:54.637362 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0307dfa9-f5fe-450a-9ffd-4aef20fc1132 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:10:54.648552 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28431d6e-09ab-44f5-a6a3-91718dea30ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:10:54.649358 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28431d6e-09ab-44f5-a6a3-91718dea30ea None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:54.649876 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28431d6e-09ab-44f5-a6a3-91718dea30ea None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:54.650246 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28431d6e-09ab-44f5-a6a3-91718dea30ea None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:54.651068 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28431d6e-09ab-44f5-a6a3-91718dea30ea None admin] RBAC: Adding query filter params (group.id=a53b135c5f244755bca9b0669c51acd8, role.id=0679d6628a25438d90dc4bdc0401eda2, scope.project.id=bd9c06e0f78e4d26902fd521a2fe013e) {{(pid=81449) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:10:54.651476 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28431d6e-09ab-44f5-a6a3-91718dea30ea None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:10:54.653025 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28431d6e-09ab-44f5-a6a3-91718dea30ea None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:10:54.661461 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 183/368] 10.222.0.15 () {56 vars in 1376 bytes} [Thu Sep 15 12:10:54 2022] GET /identity/v3/role_assignments?group.id=a53b135c5f244755bca9b0669c51acd8&scope.project.id=bd9c06e0f78e4d26902fd521a2fe013e&role.id=0679d6628a25438d90dc4bdc0401eda2 => generated 260 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:10:54.681637 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fa8514-e11e-445a-abbc-846ceab518bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:10:54.682550 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fa8514-e11e-445a-abbc-846ceab518bd None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:54.682998 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fa8514-e11e-445a-abbc-846ceab518bd None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:54.683417 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fa8514-e11e-445a-abbc-846ceab518bd None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:54.696987 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28fa8514-e11e-445a-abbc-846ceab518bd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 12:10:54.702601 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application [None req-28fa8514-e11e-445a-abbc-846ceab518bd None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 15 12:10:54.702601 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:10:54.702601 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:10:54.702601 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:10:54.702601 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:10:54.702601 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:10:54.702601 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:10:54.702601 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:10:54.702601 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:10:54.702601 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:10:54.702601 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:10:54.702601 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:10:54.702601 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 12:10:54.702601 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 12:10:54.702601 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 12:10:54.702601 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 12:10:54.703778 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 12:10:54.703778 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 12:10:54.703778 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 12:10:54.703778 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 12:10:54.703778 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:10:54.703778 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:10:54.703778 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:10:54.703778 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:10:54.703778 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:10:54.703778 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:10:54.703778 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:10:54.703778 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application with Lock( Sep 15 12:10:54.703778 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:10:54.703778 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:10:54.703778 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:10:54.703778 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:10:54.704846 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:10:54.704846 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:10:54.704846 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:10:54.704846 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:10:54.704846 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 12:10:54.704846 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 12:10:54.704846 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 12:10:54.704846 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 12:10:54.704846 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 12:10:54.704846 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 12:10:54.704846 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 15 12:10:54.704846 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 15 12:10:54.709991 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 186/369] 10.222.0.15 () {56 vars in 1114 bytes} [Thu Sep 15 12:10:54 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 15 12:10:54.714563 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28431d6e-09ab-44f5-a6a3-91718dea30ea None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:10:54.818287 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:10:54.826538 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fa8514-e11e-445a-abbc-846ceab518bd None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:10:54.860848 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28431d6e-09ab-44f5-a6a3-91718dea30ea None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:10:54.931305 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a23d6f-9790-43dc-8d99-6406dfbd6078 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:10:54.932792 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a23d6f-9790-43dc-8d99-6406dfbd6078 None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:54.935679 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a23d6f-9790-43dc-8d99-6406dfbd6078 None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:54.936117 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a23d6f-9790-43dc-8d99-6406dfbd6078 None admin] PATH_INFO: `/v3/roles` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:54.936968 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a23d6f-9790-43dc-8d99-6406dfbd6078 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81449) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:10:54.937443 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a23d6f-9790-43dc-8d99-6406dfbd6078 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:10:54.938972 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a23d6f-9790-43dc-8d99-6406dfbd6078 None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:10:54.951379 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 184/370] 10.222.0.15 () {56 vars in 1111 bytes} [Thu Sep 15 12:10:54 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:10:54.958014 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a23d6f-9790-43dc-8d99-6406dfbd6078 None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:10:55.002395 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fa8514-e11e-445a-abbc-846ceab518bd None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:10:55.065368 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a789ebf9-df8d-4570-b90f-3bc46583bcc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:10:55.067096 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a789ebf9-df8d-4570-b90f-3bc46583bcc8 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:55.067384 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a789ebf9-df8d-4570-b90f-3bc46583bcc8 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:55.067658 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a789ebf9-df8d-4570-b90f-3bc46583bcc8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:55.089135 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a789ebf9-df8d-4570-b90f-3bc46583bcc8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 12:10:55.099895 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application [None req-a789ebf9-df8d-4570-b90f-3bc46583bcc8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 15 12:10:55.099895 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:10:55.099895 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:10:55.099895 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:10:55.099895 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:10:55.099895 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:10:55.099895 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:10:55.099895 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:10:55.099895 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:10:55.099895 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:10:55.099895 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:10:55.099895 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:10:55.099895 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 12:10:55.099895 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 12:10:55.099895 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 12:10:55.099895 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 12:10:55.100822 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 12:10:55.100822 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 12:10:55.100822 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 12:10:55.100822 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 12:10:55.100822 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:10:55.100822 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:10:55.100822 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:10:55.100822 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:10:55.100822 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:10:55.100822 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:10:55.100822 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:10:55.100822 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application with Lock( Sep 15 12:10:55.100822 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:10:55.100822 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:10:55.100822 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:10:55.100822 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:10:55.101614 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:10:55.101614 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:10:55.101614 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:10:55.101614 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:10:55.101614 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 12:10:55.101614 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 12:10:55.101614 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 12:10:55.101614 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 12:10:55.101614 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 12:10:55.101614 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 12:10:55.101614 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 15 12:10:55.101614 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 15 12:10:55.101614 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 187/371] 10.222.0.15 () {56 vars in 1096 bytes} [Thu Sep 15 12:10:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 15 12:10:55.107132 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a789ebf9-df8d-4570-b90f-3bc46583bcc8 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:10:55.121281 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:10:55.151841 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a23d6f-9790-43dc-8d99-6406dfbd6078 None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:10:55.233665 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2903c119-1703-4c4b-93bb-5cc7aa0079df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:10:55.234387 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2903c119-1703-4c4b-93bb-5cc7aa0079df None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:55.234542 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2903c119-1703-4c4b-93bb-5cc7aa0079df None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:55.234731 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2903c119-1703-4c4b-93bb-5cc7aa0079df None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:55.268141 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:10:55.273225 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a789ebf9-df8d-4570-b90f-3bc46583bcc8 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:10:55.335090 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa93688f-a560-4fef-b649-d465cdc69053 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:10:55.335708 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa93688f-a560-4fef-b649-d465cdc69053 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:55.337076 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa93688f-a560-4fef-b649-d465cdc69053 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:55.337354 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa93688f-a560-4fef-b649-d465cdc69053 None admin] PATH_INFO: `/v3/roles` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:55.338029 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa93688f-a560-4fef-b649-d465cdc69053 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:10:55.338566 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa93688f-a560-4fef-b649-d465cdc69053 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:10:55.339946 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa93688f-a560-4fef-b649-d465cdc69053 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:10:55.354468 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 188/372] 10.222.0.15 () {56 vars in 1099 bytes} [Thu Sep 15 12:10:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:10:55.361007 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa93688f-a560-4fef-b649-d465cdc69053 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:10:55.366127 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2903c119-1703-4c4b-93bb-5cc7aa0079df None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:10:55.367611 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2903c119-1703-4c4b-93bb-5cc7aa0079df None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:10:55.511893 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 185/373] 10.222.0.15 () {58 vars in 1282 bytes} [Thu Sep 15 12:10:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 554 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 15 12:10:55.517219 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2903c119-1703-4c4b-93bb-5cc7aa0079df None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:10:55.519397 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:10:55.577573 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa93688f-a560-4fef-b649-d465cdc69053 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:10:55.602827 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:10:55.640138 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac3fe86-8bd0-4a8b-ba41-3bf121e67004 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:10:55.640780 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac3fe86-8bd0-4a8b-ba41-3bf121e67004 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:55.641073 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac3fe86-8bd0-4a8b-ba41-3bf121e67004 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:55.641348 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac3fe86-8bd0-4a8b-ba41-3bf121e67004 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:55.698898 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2903c119-1703-4c4b-93bb-5cc7aa0079df None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:10:55.752571 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac3fe86-8bd0-4a8b-ba41-3bf121e67004 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:10:55.754372 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac3fe86-8bd0-4a8b-ba41-3bf121e67004 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:10:55.770633 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7884b308-936f-4e39-83a8-a6dfef1ca8f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:10:55.771317 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7884b308-936f-4e39-83a8-a6dfef1ca8f5 None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:55.772115 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7884b308-936f-4e39-83a8-a6dfef1ca8f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:55.772416 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7884b308-936f-4e39-83a8-a6dfef1ca8f5 None admin] PATH_INFO: `/v3/projects/bd9c06e0f78e4d26902fd521a2fe013e` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:55.773784 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7884b308-936f-4e39-83a8-a6dfef1ca8f5 None admin] RBAC: Authorizing `identity:get_project(project_id=bd9c06e0f78e4d26902fd521a2fe013e)` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:10:55.775377 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7884b308-936f-4e39-83a8-a6dfef1ca8f5 None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:10:55.777545 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 186/374] 10.222.0.15 () {56 vars in 1186 bytes} [Thu Sep 15 12:10:55 2022] GET /identity/v3/projects/bd9c06e0f78e4d26902fd521a2fe013e => generated 302 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:10:55.783282 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7884b308-936f-4e39-83a8-a6dfef1ca8f5 None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:10:55.861117 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:10:55.949663 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 189/375] 10.222.0.15 () {58 vars in 1282 bytes} [Thu Sep 15 12:10:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 589 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 15 12:10:55.953435 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac3fe86-8bd0-4a8b-ba41-3bf121e67004 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:10:55.972411 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7884b308-936f-4e39-83a8-a6dfef1ca8f5 None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:10:55.992907 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:10:56.040145 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ee5c26-8003-4d53-9b2d-6037da046ad5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:10:56.041150 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ee5c26-8003-4d53-9b2d-6037da046ad5 None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:56.041624 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ee5c26-8003-4d53-9b2d-6037da046ad5 None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:56.042105 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ee5c26-8003-4d53-9b2d-6037da046ad5 None admin] PATH_INFO: `/v3/groups/a53b135c5f244755bca9b0669c51acd8` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:56.043854 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ee5c26-8003-4d53-9b2d-6037da046ad5 None admin] RBAC: Authorizing `identity:get_group(group_id=a53b135c5f244755bca9b0669c51acd8)` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:10:56.045500 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ee5c26-8003-4d53-9b2d-6037da046ad5 None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:10:56.051448 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 187/376] 10.222.0.15 () {56 vars in 1180 bytes} [Thu Sep 15 12:10:55 2022] GET /identity/v3/groups/a53b135c5f244755bca9b0669c51acd8 => generated 226 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 12:10:56.074657 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ee5c26-8003-4d53-9b2d-6037da046ad5 None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:10:56.171919 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac3fe86-8bd0-4a8b-ba41-3bf121e67004 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:10:56.244113 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc39feb-3fd6-4d0f-88ee-7060a2a5de67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:10:56.245259 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc39feb-3fd6-4d0f-88ee-7060a2a5de67 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:56.246036 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc39feb-3fd6-4d0f-88ee-7060a2a5de67 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:56.246522 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc39feb-3fd6-4d0f-88ee-7060a2a5de67 None admin] PATH_INFO: `/v3/projects/69bea7b802b04b689af0d31b020f8554` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:56.248692 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc39feb-3fd6-4d0f-88ee-7060a2a5de67 None admin] RBAC: Authorizing `identity:get_project(project_id=69bea7b802b04b689af0d31b020f8554)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:10:56.250526 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc39feb-3fd6-4d0f-88ee-7060a2a5de67 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:10:56.255601 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 190/377] 10.222.0.15 () {56 vars in 1186 bytes} [Thu Sep 15 12:10:55 2022] GET /identity/v3/projects/69bea7b802b04b689af0d31b020f8554 => generated 344 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:10:56.267773 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc39feb-3fd6-4d0f-88ee-7060a2a5de67 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:10:56.285960 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ee5c26-8003-4d53-9b2d-6037da046ad5 None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:10:56.316954 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:10:56.341345 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c56429-46d4-433f-b537-6dd6223133de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:10:56.341846 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c56429-46d4-433f-b537-6dd6223133de None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:56.342126 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c56429-46d4-433f-b537-6dd6223133de None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:56.342392 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c56429-46d4-433f-b537-6dd6223133de None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:56.343113 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c56429-46d4-433f-b537-6dd6223133de None admin] RBAC: Adding query filter params (group.id=a53b135c5f244755bca9b0669c51acd8, role.id=1081b93ddfd547649ba8582f7a6e719e, scope.project.id=bd9c06e0f78e4d26902fd521a2fe013e) {{(pid=81449) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:10:56.343415 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c56429-46d4-433f-b537-6dd6223133de None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:10:56.344868 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13c56429-46d4-433f-b537-6dd6223133de None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:10:56.352140 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 188/378] 10.222.0.15 () {56 vars in 1376 bytes} [Thu Sep 15 12:10:56 2022] GET /identity/v3/role_assignments?group.id=a53b135c5f244755bca9b0669c51acd8&scope.project.id=bd9c06e0f78e4d26902fd521a2fe013e&role.id=1081b93ddfd547649ba8582f7a6e719e => generated 260 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 12:10:56.353610 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5e4379-9040-48a2-90ac-f870455ebfa9 None None] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:56.353831 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5e4379-9040-48a2-90ac-f870455ebfa9 None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:56.354103 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5e4379-9040-48a2-90ac-f870455ebfa9 None None] PATH_INFO: `/` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:56.355782 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 189/379] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 12:10:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:10:56.362639 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9327d3ec-7954-43f3-b092-20c310feebfe None None] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:56.362937 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9327d3ec-7954-43f3-b092-20c310feebfe None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:56.363189 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9327d3ec-7954-43f3-b092-20c310feebfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:56.386288 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.auth.core [None req-9327d3ec-7954-43f3-b092-20c310feebfe None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81449) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:10:56.387551 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-9327d3ec-7954-43f3-b092-20c310feebfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:10:56.420881 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc39feb-3fd6-4d0f-88ee-7060a2a5de67 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:10:56.447465 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:10:56.504681 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16bba1a1-62dc-44ce-b204-a0fdc24bfe25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:10:56.506977 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bba1a1-62dc-44ce-b204-a0fdc24bfe25 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:56.507663 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bba1a1-62dc-44ce-b204-a0fdc24bfe25 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:56.508749 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bba1a1-62dc-44ce-b204-a0fdc24bfe25 None admin] PATH_INFO: `/v3/groups/808ddc3a7fe547139412ee50b7269ea7` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:56.515622 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 190/380] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 12:10:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 12:10:56.522887 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16bba1a1-62dc-44ce-b204-a0fdc24bfe25 None admin] RBAC: Authorizing `identity:get_group(group_id=808ddc3a7fe547139412ee50b7269ea7)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:10:56.524704 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16bba1a1-62dc-44ce-b204-a0fdc24bfe25 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:10:56.526962 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 191/381] 10.222.0.15 () {56 vars in 1180 bytes} [Thu Sep 15 12:10:56 2022] GET /identity/v3/groups/808ddc3a7fe547139412ee50b7269ea7 => generated 229 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 12:10:56.532614 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9327d3ec-7954-43f3-b092-20c310feebfe None None] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:10:56.667768 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9327d3ec-7954-43f3-b092-20c310feebfe None None] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:10:56.710485 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2609f800-1f7b-46c7-9a90-0bac705a76b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:10:56.711108 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2609f800-1f7b-46c7-9a90-0bac705a76b6 None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:56.711384 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2609f800-1f7b-46c7-9a90-0bac705a76b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:56.711653 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2609f800-1f7b-46c7-9a90-0bac705a76b6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:56.712381 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2609f800-1f7b-46c7-9a90-0bac705a76b6 None admin] RBAC: Adding query filter params (group.id=808ddc3a7fe547139412ee50b7269ea7, role.id=9c769b5c82884766b2c510b1f0f1364c, scope.project.id=69bea7b802b04b689af0d31b020f8554) {{(pid=81449) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:10:56.712688 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2609f800-1f7b-46c7-9a90-0bac705a76b6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:10:56.716082 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2609f800-1f7b-46c7-9a90-0bac705a76b6 None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:10:56.723252 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 191/382] 10.222.0.15 () {56 vars in 1376 bytes} [Thu Sep 15 12:10:56 2022] GET /identity/v3/role_assignments?group.id=808ddc3a7fe547139412ee50b7269ea7&scope.project.id=69bea7b802b04b689af0d31b020f8554&role.id=9c769b5c82884766b2c510b1f0f1364c => generated 260 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:10:56.820208 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16bba1a1-62dc-44ce-b204-a0fdc24bfe25 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:10:56.939845 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:10:56.944782 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16bba1a1-62dc-44ce-b204-a0fdc24bfe25 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:10:56.991495 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b770eba-340a-4260-bb3b-0328176a31ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:10:56.992478 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b770eba-340a-4260-bb3b-0328176a31ce None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:56.992991 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b770eba-340a-4260-bb3b-0328176a31ce None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:56.993567 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b770eba-340a-4260-bb3b-0328176a31ce None admin] PATH_INFO: `/v3/roles/member` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:57.006893 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b770eba-340a-4260-bb3b-0328176a31ce None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 12:10:57.012118 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application [None req-4b770eba-340a-4260-bb3b-0328176a31ce None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 15 12:10:57.012118 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:10:57.012118 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:10:57.012118 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:10:57.012118 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:10:57.012118 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:10:57.012118 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:10:57.012118 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:10:57.012118 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:10:57.012118 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:10:57.012118 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:10:57.012118 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:10:57.012118 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 12:10:57.012118 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 12:10:57.012118 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 12:10:57.012118 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 12:10:57.013098 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 12:10:57.013098 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 12:10:57.013098 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 12:10:57.013098 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 12:10:57.013098 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:10:57.013098 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:10:57.013098 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:10:57.013098 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:10:57.013098 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:10:57.013098 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:10:57.013098 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:10:57.013098 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application with Lock( Sep 15 12:10:57.013098 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:10:57.013098 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:10:57.013098 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:10:57.013098 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:10:57.013942 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:10:57.013942 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:10:57.013942 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:10:57.013942 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:10:57.013942 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 12:10:57.013942 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 12:10:57.013942 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 12:10:57.013942 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 12:10:57.013942 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 12:10:57.013942 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 12:10:57.013942 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 15 12:10:57.013942 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 15 12:10:57.016525 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 192/383] 10.222.0.15 () {56 vars in 1099 bytes} [Thu Sep 15 12:10:56 2022] GET /identity/v3/roles/member => generated 84 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:10:57.025850 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2609f800-1f7b-46c7-9a90-0bac705a76b6 None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:10:57.100064 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:10:57.175556 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2609f800-1f7b-46c7-9a90-0bac705a76b6 None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:10:57.214161 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b33a858-330b-4f28-b9ca-4ecca16913ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:10:57.214711 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b33a858-330b-4f28-b9ca-4ecca16913ee None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:57.214988 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b33a858-330b-4f28-b9ca-4ecca16913ee None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:57.215262 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b33a858-330b-4f28-b9ca-4ecca16913ee None admin] PATH_INFO: `/v3/roles` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:57.215951 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b33a858-330b-4f28-b9ca-4ecca16913ee None admin] RBAC: Adding query filter params (name=member) {{(pid=81449) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:10:57.216252 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b33a858-330b-4f28-b9ca-4ecca16913ee None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:10:57.217728 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b33a858-330b-4f28-b9ca-4ecca16913ee None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:10:57.234727 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 192/384] 10.222.0.15 () {56 vars in 1101 bytes} [Thu Sep 15 12:10:57 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:10:57.234938 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374a3ab0-5e24-4faa-8e0a-f1deee8e9fd0 None None] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:57.235607 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374a3ab0-5e24-4faa-8e0a-f1deee8e9fd0 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:57.236114 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374a3ab0-5e24-4faa-8e0a-f1deee8e9fd0 None None] PATH_INFO: `/` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:57.237783 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 193/385] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 12:10:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 15 12:10:57.240894 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b33a858-330b-4f28-b9ca-4ecca16913ee None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:10:57.245482 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab329207-b9ac-4bd8-aa10-c55fd36ea6eb None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:57.245805 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab329207-b9ac-4bd8-aa10-c55fd36ea6eb None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:57.246043 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab329207-b9ac-4bd8-aa10-c55fd36ea6eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:57.281781 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-ab329207-b9ac-4bd8-aa10-c55fd36ea6eb None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:10:57.283007 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-ab329207-b9ac-4bd8-aa10-c55fd36ea6eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:10:57.370344 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 194/386] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 12:10:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 12:10:57.371819 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3989ae-590d-45ad-9535-ba26ec115dce None None] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:57.372111 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3989ae-590d-45ad-9535-ba26ec115dce None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:57.372383 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3989ae-590d-45ad-9535-ba26ec115dce None None] PATH_INFO: `/` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:57.373877 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 195/387] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 12:10:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:10:57.380476 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627ec395-0f6b-4769-8015-8fc982d9090b None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:57.380779 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627ec395-0f6b-4769-8015-8fc982d9090b None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:57.381089 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627ec395-0f6b-4769-8015-8fc982d9090b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:57.414036 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-627ec395-0f6b-4769-8015-8fc982d9090b None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:10:57.415256 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-627ec395-0f6b-4769-8015-8fc982d9090b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:10:57.428943 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b33a858-330b-4f28-b9ca-4ecca16913ee None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:10:57.441102 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:10:57.495981 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871527b9-beda-44d5-9c76-9d0bf3c692a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:10:57.496507 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871527b9-beda-44d5-9c76-9d0bf3c692a7 None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:57.496813 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871527b9-beda-44d5-9c76-9d0bf3c692a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:57.497093 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871527b9-beda-44d5-9c76-9d0bf3c692a7 None admin] PATH_INFO: `/v3/groups/a53b135c5f244755bca9b0669c51acd8` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:57.498216 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-871527b9-beda-44d5-9c76-9d0bf3c692a7 None admin] RBAC: Authorizing `identity:get_group(group_id=a53b135c5f244755bca9b0669c51acd8)` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:10:57.499664 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-871527b9-beda-44d5-9c76-9d0bf3c692a7 None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:10:57.504862 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 193/388] 10.222.0.15 () {56 vars in 1180 bytes} [Thu Sep 15 12:10:57 2022] GET /identity/v3/groups/a53b135c5f244755bca9b0669c51acd8 => generated 226 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:10:57.511020 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871527b9-beda-44d5-9c76-9d0bf3c692a7 None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:10:57.530481 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 196/389] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 12:10:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 15 12:10:57.645368 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871527b9-beda-44d5-9c76-9d0bf3c692a7 None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:10:57.674935 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627ec395-0f6b-4769-8015-8fc982d9090b None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:10:57.682510 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07bbfa82-b5ed-47aa-9ab2-c069216329cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:10:57.683464 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bbfa82-b5ed-47aa-9ab2-c069216329cc None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:57.683938 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bbfa82-b5ed-47aa-9ab2-c069216329cc None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:57.684421 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07bbfa82-b5ed-47aa-9ab2-c069216329cc None admin] PATH_INFO: `/v3/projects/10a21ffb0b8d4800a805c76f171c40c1` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:57.686244 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07bbfa82-b5ed-47aa-9ab2-c069216329cc None admin] RBAC: Authorizing `identity:get_project(project_id=10a21ffb0b8d4800a805c76f171c40c1)` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:10:57.688305 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07bbfa82-b5ed-47aa-9ab2-c069216329cc None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:10:57.691445 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 194/390] 10.222.0.15 () {56 vars in 1186 bytes} [Thu Sep 15 12:10:57 2022] GET /identity/v3/projects/10a21ffb0b8d4800a805c76f171c40c1 => generated 298 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:10:57.697737 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07bbfa82-b5ed-47aa-9ab2-c069216329cc None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:10:57.705138 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:10:57.745977 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:10:57.814295 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07bbfa82-b5ed-47aa-9ab2-c069216329cc None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:10:57.853408 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0636fd-9915-4e78-834d-620c30537a2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:10:57.854344 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0636fd-9915-4e78-834d-620c30537a2e None admin] REQUEST_METHOD: `PUT` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:57.854625 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0636fd-9915-4e78-834d-620c30537a2e None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:57.855162 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0636fd-9915-4e78-834d-620c30537a2e None admin] PATH_INFO: `/v3/projects/10a21ffb0b8d4800a805c76f171c40c1/groups/a53b135c5f244755bca9b0669c51acd8/roles/0679d6628a25438d90dc4bdc0401eda2` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:57.857124 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0636fd-9915-4e78-834d-620c30537a2e None admin] RBAC: Authorizing `identity:create_grant(project_id=10a21ffb0b8d4800a805c76f171c40c1, group_id=a53b135c5f244755bca9b0669c51acd8, role_id=0679d6628a25438d90dc4bdc0401eda2)` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:10:57.858700 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0636fd-9915-4e78-834d-620c30537a2e None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:10:57.875977 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 195/391] 10.222.0.15 () {58 vars in 1442 bytes} [Thu Sep 15 12:10:57 2022] PUT /identity/v3/projects/10a21ffb0b8d4800a805c76f171c40c1/groups/a53b135c5f244755bca9b0669c51acd8/roles/0679d6628a25438d90dc4bdc0401eda2 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 12:10:57.911733 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0636fd-9915-4e78-834d-620c30537a2e None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:10:57.926412 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627ec395-0f6b-4769-8015-8fc982d9090b None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:10:58.049510 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42139792-b10f-4701-9a23-d9f3a2b9f391 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:10:58.050354 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42139792-b10f-4701-9a23-d9f3a2b9f391 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:58.050490 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42139792-b10f-4701-9a23-d9f3a2b9f391 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:58.050705 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42139792-b10f-4701-9a23-d9f3a2b9f391 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:58.053699 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:10:58.064293 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0636fd-9915-4e78-834d-620c30537a2e None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:10:58.085898 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42139792-b10f-4701-9a23-d9f3a2b9f391 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 12:10:58.098087 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application [None req-42139792-b10f-4701-9a23-d9f3a2b9f391 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 15 12:10:58.098087 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:10:58.098087 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:10:58.098087 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:10:58.098087 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:10:58.098087 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:10:58.098087 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:10:58.098087 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:10:58.098087 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:10:58.098087 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:10:58.098087 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:10:58.098087 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:10:58.098087 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 12:10:58.098087 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 12:10:58.098087 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 12:10:58.098087 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 12:10:58.099223 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 12:10:58.099223 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 12:10:58.099223 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 12:10:58.099223 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 12:10:58.099223 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:10:58.099223 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:10:58.099223 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:10:58.099223 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:10:58.099223 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:10:58.099223 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:10:58.099223 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:10:58.099223 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application with Lock( Sep 15 12:10:58.099223 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:10:58.099223 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:10:58.099223 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:10:58.099223 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:10:58.100246 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:10:58.100246 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:10:58.100246 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:10:58.100246 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:10:58.100246 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 12:10:58.100246 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 12:10:58.100246 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 12:10:58.100246 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 12:10:58.100246 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 12:10:58.100246 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 12:10:58.100246 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 15 12:10:58.100246 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 15 12:10:58.105287 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 197/392] 10.222.0.15 () {56 vars in 1114 bytes} [Thu Sep 15 12:10:57 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 431 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:10:58.110674 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42139792-b10f-4701-9a23-d9f3a2b9f391 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:10:58.149562 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e765a06a-436a-4e15-92c8-070e3a28ce76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:10:58.149770 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e765a06a-436a-4e15-92c8-070e3a28ce76 None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:58.149937 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e765a06a-436a-4e15-92c8-070e3a28ce76 None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:58.150191 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e765a06a-436a-4e15-92c8-070e3a28ce76 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:58.164550 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e765a06a-436a-4e15-92c8-070e3a28ce76 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 12:10:58.172919 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application [None req-e765a06a-436a-4e15-92c8-070e3a28ce76 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 15 12:10:58.172919 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:10:58.172919 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:10:58.172919 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:10:58.172919 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:10:58.172919 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:10:58.172919 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:10:58.172919 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:10:58.172919 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:10:58.172919 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:10:58.172919 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:10:58.172919 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:10:58.172919 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 12:10:58.172919 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 12:10:58.172919 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 12:10:58.172919 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 12:10:58.173918 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 12:10:58.173918 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 12:10:58.173918 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 12:10:58.173918 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 12:10:58.173918 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:10:58.173918 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:10:58.173918 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:10:58.173918 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:10:58.173918 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:10:58.173918 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:10:58.173918 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:10:58.173918 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application with Lock( Sep 15 12:10:58.173918 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:10:58.173918 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:10:58.173918 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:10:58.173918 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:10:58.174713 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:10:58.174713 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:10:58.174713 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:10:58.174713 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:10:58.174713 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 12:10:58.174713 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 12:10:58.174713 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 12:10:58.174713 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 12:10:58.174713 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 12:10:58.174713 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 12:10:58.174713 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 15 12:10:58.174713 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application Sep 15 12:10:58.174713 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 196/393] 10.222.0.15 () {56 vars in 1099 bytes} [Thu Sep 15 12:10:57 2022] GET /identity/v3/roles/member => generated 84 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:10:58.186025 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e765a06a-436a-4e15-92c8-070e3a28ce76 None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:10:58.228058 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:10:58.250732 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42139792-b10f-4701-9a23-d9f3a2b9f391 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:10:58.299371 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb2695d-873d-44d1-9a84-363eb6c8a1fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:10:58.300225 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb2695d-873d-44d1-9a84-363eb6c8a1fd None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:58.300832 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb2695d-873d-44d1-9a84-363eb6c8a1fd None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:58.301391 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb2695d-873d-44d1-9a84-363eb6c8a1fd None admin] PATH_INFO: `/v3/roles` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:58.302236 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb2695d-873d-44d1-9a84-363eb6c8a1fd None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:10:58.302734 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb2695d-873d-44d1-9a84-363eb6c8a1fd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:10:58.304562 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb2695d-873d-44d1-9a84-363eb6c8a1fd None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:10:58.314538 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 198/394] 10.222.0.15 () {56 vars in 1111 bytes} [Thu Sep 15 12:10:58 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 12:10:58.322013 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb2695d-873d-44d1-9a84-363eb6c8a1fd None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:10:58.441891 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb2695d-873d-44d1-9a84-363eb6c8a1fd None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:10:58.471708 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e765a06a-436a-4e15-92c8-070e3a28ce76 None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:10:58.482591 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90a0e25-424b-47cf-a040-a0905755be56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:10:58.484121 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90a0e25-424b-47cf-a040-a0905755be56 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:58.484775 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90a0e25-424b-47cf-a040-a0905755be56 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:58.485209 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90a0e25-424b-47cf-a040-a0905755be56 None admin] PATH_INFO: `/v3/groups/a53b135c5f244755bca9b0669c51acd8` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:58.487196 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90a0e25-424b-47cf-a040-a0905755be56 None admin] RBAC: Authorizing `identity:get_group(group_id=a53b135c5f244755bca9b0669c51acd8)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:10:58.488928 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90a0e25-424b-47cf-a040-a0905755be56 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:10:58.492037 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 199/395] 10.222.0.15 () {56 vars in 1180 bytes} [Thu Sep 15 12:10:58 2022] GET /identity/v3/groups/a53b135c5f244755bca9b0669c51acd8 => generated 226 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:10:58.499263 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90a0e25-424b-47cf-a040-a0905755be56 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:10:58.529177 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:10:58.566048 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52938059-7b4b-484a-b13b-5126c8021708 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:10:58.566914 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52938059-7b4b-484a-b13b-5126c8021708 None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:58.567218 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52938059-7b4b-484a-b13b-5126c8021708 None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:58.567607 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52938059-7b4b-484a-b13b-5126c8021708 None admin] PATH_INFO: `/v3/roles` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:58.568380 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52938059-7b4b-484a-b13b-5126c8021708 None admin] RBAC: Adding query filter params (name=member) {{(pid=81449) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:10:58.569328 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52938059-7b4b-484a-b13b-5126c8021708 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:10:58.570818 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52938059-7b4b-484a-b13b-5126c8021708 None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:10:58.592349 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 197/396] 10.222.0.15 () {56 vars in 1101 bytes} [Thu Sep 15 12:10:58 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 407 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:10:58.597626 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52938059-7b4b-484a-b13b-5126c8021708 None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:10:58.612707 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90a0e25-424b-47cf-a040-a0905755be56 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:10:58.640451 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:10:58.651665 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc90751-8c7d-4c0a-9daa-33a3e4a8bb04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:10:58.652659 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc90751-8c7d-4c0a-9daa-33a3e4a8bb04 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:58.653165 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc90751-8c7d-4c0a-9daa-33a3e4a8bb04 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:58.653723 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc90751-8c7d-4c0a-9daa-33a3e4a8bb04 None admin] PATH_INFO: `/v3/projects/10a21ffb0b8d4800a805c76f171c40c1` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:58.655225 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc90751-8c7d-4c0a-9daa-33a3e4a8bb04 None admin] RBAC: Authorizing `identity:get_project(project_id=10a21ffb0b8d4800a805c76f171c40c1)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:10:58.656967 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc90751-8c7d-4c0a-9daa-33a3e4a8bb04 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:10:58.659139 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 200/397] 10.222.0.15 () {56 vars in 1186 bytes} [Thu Sep 15 12:10:58 2022] GET /identity/v3/projects/10a21ffb0b8d4800a805c76f171c40c1 => generated 298 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 12:10:58.683665 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc90751-8c7d-4c0a-9daa-33a3e4a8bb04 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:10:58.773288 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52938059-7b4b-484a-b13b-5126c8021708 None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:10:58.813821 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5530882b-d90a-43c1-9b45-0f2d33cbd34c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:10:58.814466 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5530882b-d90a-43c1-9b45-0f2d33cbd34c None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:58.814746 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5530882b-d90a-43c1-9b45-0f2d33cbd34c None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:58.815018 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5530882b-d90a-43c1-9b45-0f2d33cbd34c None admin] PATH_INFO: `/v3/groups/a53b135c5f244755bca9b0669c51acd8` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:58.816747 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5530882b-d90a-43c1-9b45-0f2d33cbd34c None admin] RBAC: Authorizing `identity:get_group(group_id=a53b135c5f244755bca9b0669c51acd8)` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:10:58.818194 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5530882b-d90a-43c1-9b45-0f2d33cbd34c None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:10:58.822750 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 198/398] 10.222.0.15 () {56 vars in 1180 bytes} [Thu Sep 15 12:10:58 2022] GET /identity/v3/groups/a53b135c5f244755bca9b0669c51acd8 => generated 226 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:10:58.827227 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5530882b-d90a-43c1-9b45-0f2d33cbd34c None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:10:58.853153 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:10:58.895784 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc90751-8c7d-4c0a-9daa-33a3e4a8bb04 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:10:58.947163 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ec4044-e48c-4b94-84fe-d39c1ee4266b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:10:58.947963 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ec4044-e48c-4b94-84fe-d39c1ee4266b None admin] REQUEST_METHOD: `PUT` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:58.948270 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ec4044-e48c-4b94-84fe-d39c1ee4266b None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:58.948783 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ec4044-e48c-4b94-84fe-d39c1ee4266b None admin] PATH_INFO: `/v3/projects/10a21ffb0b8d4800a805c76f171c40c1/groups/a53b135c5f244755bca9b0669c51acd8/roles/1081b93ddfd547649ba8582f7a6e719e` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:58.950995 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ec4044-e48c-4b94-84fe-d39c1ee4266b None admin] RBAC: Authorizing `identity:create_grant(project_id=10a21ffb0b8d4800a805c76f171c40c1, group_id=a53b135c5f244755bca9b0669c51acd8, role_id=1081b93ddfd547649ba8582f7a6e719e)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:10:58.952431 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9ec4044-e48c-4b94-84fe-d39c1ee4266b None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:10:58.981175 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 201/399] 10.222.0.15 () {58 vars in 1442 bytes} [Thu Sep 15 12:10:58 2022] PUT /identity/v3/projects/10a21ffb0b8d4800a805c76f171c40c1/groups/a53b135c5f244755bca9b0669c51acd8/roles/1081b93ddfd547649ba8582f7a6e719e => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 12:10:58.992109 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:10:59.010353 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5530882b-d90a-43c1-9b45-0f2d33cbd34c None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:10:59.068260 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704ee73b-d868-491b-af77-2138bf5437b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:10:59.068881 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704ee73b-d868-491b-af77-2138bf5437b1 None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:59.069160 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704ee73b-d868-491b-af77-2138bf5437b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:59.069426 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704ee73b-d868-491b-af77-2138bf5437b1 None admin] PATH_INFO: `/v3/projects/bd9c06e0f78e4d26902fd521a2fe013e` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:59.070652 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-704ee73b-d868-491b-af77-2138bf5437b1 None admin] RBAC: Authorizing `identity:get_project(project_id=bd9c06e0f78e4d26902fd521a2fe013e)` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:10:59.072118 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-704ee73b-d868-491b-af77-2138bf5437b1 None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:10:59.074390 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 199/400] 10.222.0.15 () {56 vars in 1186 bytes} [Thu Sep 15 12:10:58 2022] GET /identity/v3/projects/bd9c06e0f78e4d26902fd521a2fe013e => generated 302 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:10:59.080140 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ec4044-e48c-4b94-84fe-d39c1ee4266b None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:10:59.214171 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ec4044-e48c-4b94-84fe-d39c1ee4266b None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:10:59.259598 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682f9029-fab6-432f-8892-5cb06b95db3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:10:59.260367 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682f9029-fab6-432f-8892-5cb06b95db3b None admin] REQUEST_METHOD: `PUT` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:59.260556 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682f9029-fab6-432f-8892-5cb06b95db3b None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:59.260866 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682f9029-fab6-432f-8892-5cb06b95db3b None admin] PATH_INFO: `/v3/projects/bd9c06e0f78e4d26902fd521a2fe013e/groups/a53b135c5f244755bca9b0669c51acd8/roles/0679d6628a25438d90dc4bdc0401eda2` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:59.266234 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682f9029-fab6-432f-8892-5cb06b95db3b None admin] RBAC: Authorizing `identity:create_grant(project_id=bd9c06e0f78e4d26902fd521a2fe013e, group_id=a53b135c5f244755bca9b0669c51acd8, role_id=0679d6628a25438d90dc4bdc0401eda2)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:10:59.267708 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682f9029-fab6-432f-8892-5cb06b95db3b None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:10:59.289929 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:10:59.296414 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 202/401] 10.222.0.15 () {58 vars in 1442 bytes} [Thu Sep 15 12:10:59 2022] PUT /identity/v3/projects/bd9c06e0f78e4d26902fd521a2fe013e/groups/a53b135c5f244755bca9b0669c51acd8/roles/0679d6628a25438d90dc4bdc0401eda2 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:10:59.359079 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81dd621e-a59d-4533-872f-1b9657878ee3 None None] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:59.359079 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81dd621e-a59d-4533-872f-1b9657878ee3 None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:59.359079 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81dd621e-a59d-4533-872f-1b9657878ee3 None None] PATH_INFO: `/` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:59.359079 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 200/402] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 12:10:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:10:59.364964 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215f751e-8ad1-4ae1-9b47-370642ff0d7b None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:59.365372 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215f751e-8ad1-4ae1-9b47-370642ff0d7b None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:59.366089 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215f751e-8ad1-4ae1-9b47-370642ff0d7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:59.389478 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-215f751e-8ad1-4ae1-9b47-370642ff0d7b None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:10:59.390705 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-215f751e-8ad1-4ae1-9b47-370642ff0d7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:10:59.475389 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 203/403] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 12:10:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 12:10:59.667877 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7589e4e9-f228-4561-852c-cc88c1ddf906 None None] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:59.668374 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7589e4e9-f228-4561-852c-cc88c1ddf906 None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:59.668374 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7589e4e9-f228-4561-852c-cc88c1ddf906 None None] PATH_INFO: `/` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:59.669939 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 201/404] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 12:10:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:10:59.675422 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed53d73-9e6b-4f70-8f6c-c0cc50edec6a None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:10:59.675707 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed53d73-9e6b-4f70-8f6c-c0cc50edec6a None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:10:59.675979 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed53d73-9e6b-4f70-8f6c-c0cc50edec6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:10:59.701755 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-eed53d73-9e6b-4f70-8f6c-c0cc50edec6a None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:10:59.702935 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-eed53d73-9e6b-4f70-8f6c-c0cc50edec6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:10:59.779902 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 204/405] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 12:10:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 12:10:59.851101 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7589e4e9-f228-4561-852c-cc88c1ddf906 None None] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:10:59.984830 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7589e4e9-f228-4561-852c-cc88c1ddf906 None None] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:00.021000 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15c1b97-4c64-4231-95f7-576cfac6661d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:00.022596 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15c1b97-4c64-4231-95f7-576cfac6661d None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:00.023066 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15c1b97-4c64-4231-95f7-576cfac6661d None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:00.023524 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15c1b97-4c64-4231-95f7-576cfac6661d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:00.038789 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15c1b97-4c64-4231-95f7-576cfac6661d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 12:11:00.044552 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application [None req-e15c1b97-4c64-4231-95f7-576cfac6661d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 15 12:11:00.044552 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:11:00.044552 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:11:00.044552 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:11:00.044552 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:11:00.044552 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:11:00.044552 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:11:00.044552 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:11:00.044552 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:11:00.044552 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:11:00.044552 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:11:00.044552 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:11:00.044552 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 12:11:00.044552 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 12:11:00.044552 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 12:11:00.044552 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 12:11:00.045468 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 12:11:00.045468 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 12:11:00.045468 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 12:11:00.045468 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 12:11:00.045468 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:11:00.045468 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:11:00.045468 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:11:00.045468 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:11:00.045468 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:11:00.045468 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:11:00.045468 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:11:00.045468 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application with Lock( Sep 15 12:11:00.045468 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:11:00.045468 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:11:00.045468 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:11:00.045468 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:11:00.046286 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:11:00.046286 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:11:00.046286 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:11:00.046286 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:11:00.046286 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 12:11:00.046286 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 12:11:00.046286 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 12:11:00.046286 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 12:11:00.046286 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 12:11:00.046286 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 12:11:00.046286 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 15 12:11:00.046286 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application Sep 15 12:11:00.048560 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 202/406] 10.222.0.15 () {56 vars in 1114 bytes} [Thu Sep 15 12:10:59 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:11:00.056671 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed53d73-9e6b-4f70-8f6c-c0cc50edec6a None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:00.123223 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:11:00.176115 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed53d73-9e6b-4f70-8f6c-c0cc50edec6a None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:00.228734 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d158781d-555c-4bac-9970-5ecb7f1a9a46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:00.229313 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d158781d-555c-4bac-9970-5ecb7f1a9a46 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:00.229592 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d158781d-555c-4bac-9970-5ecb7f1a9a46 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:00.230132 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d158781d-555c-4bac-9970-5ecb7f1a9a46 None admin] PATH_INFO: `/v3/roles` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:00.230812 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d158781d-555c-4bac-9970-5ecb7f1a9a46 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:11:00.231112 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d158781d-555c-4bac-9970-5ecb7f1a9a46 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:11:00.232505 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d158781d-555c-4bac-9970-5ecb7f1a9a46 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:11:00.241634 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15c1b97-4c64-4231-95f7-576cfac6661d None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:00.241932 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 205/407] 10.222.0.15 () {56 vars in 1111 bytes} [Thu Sep 15 12:11:00 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:11:00.249755 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d158781d-555c-4bac-9970-5ecb7f1a9a46 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:00.262307 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:11:00.359592 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15c1b97-4c64-4231-95f7-576cfac6661d None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:00.421498 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3561cabe-7331-472e-9985-c224834a5986 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:00.422228 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3561cabe-7331-472e-9985-c224834a5986 None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:00.422425 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3561cabe-7331-472e-9985-c224834a5986 None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:00.422699 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3561cabe-7331-472e-9985-c224834a5986 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:00.424948 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d158781d-555c-4bac-9970-5ecb7f1a9a46 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:00.446689 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:11:00.448006 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3561cabe-7331-472e-9985-c224834a5986 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 12:11:00.453072 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application [None req-3561cabe-7331-472e-9985-c224834a5986 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 15 12:11:00.453072 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:11:00.453072 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:11:00.453072 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:11:00.453072 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:11:00.453072 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:11:00.453072 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:11:00.453072 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:11:00.453072 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:11:00.453072 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:11:00.453072 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:11:00.453072 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:11:00.453072 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 12:11:00.453072 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 12:11:00.453072 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 12:11:00.453072 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 12:11:00.454147 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 12:11:00.454147 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 12:11:00.454147 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 12:11:00.454147 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 12:11:00.454147 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:11:00.454147 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:11:00.454147 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:11:00.454147 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:11:00.454147 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:11:00.454147 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:11:00.454147 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:11:00.454147 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application with Lock( Sep 15 12:11:00.454147 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:11:00.454147 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:11:00.454147 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:11:00.454147 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:11:00.454948 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:11:00.454948 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:11:00.454948 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:11:00.454948 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:11:00.454948 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 12:11:00.454948 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 12:11:00.454948 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 12:11:00.454948 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 12:11:00.454948 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 12:11:00.454948 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 12:11:00.454948 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 15 12:11:00.454948 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application Sep 15 12:11:00.454948 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 203/408] 10.222.0.15 () {56 vars in 1096 bytes} [Thu Sep 15 12:11:00 2022] GET /identity/v3/roles/admin => generated 83 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:11:00.463478 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3561cabe-7331-472e-9985-c224834a5986 None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:00.473764 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1e4787-d05b-442a-9857-caf0b2d5cfef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:00.475475 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1e4787-d05b-442a-9857-caf0b2d5cfef None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:00.475955 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1e4787-d05b-442a-9857-caf0b2d5cfef None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:00.476447 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1e4787-d05b-442a-9857-caf0b2d5cfef None admin] PATH_INFO: `/v3/groups/a53b135c5f244755bca9b0669c51acd8` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:00.478030 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1e4787-d05b-442a-9857-caf0b2d5cfef None admin] RBAC: Authorizing `identity:get_group(group_id=a53b135c5f244755bca9b0669c51acd8)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:11:00.480127 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1e4787-d05b-442a-9857-caf0b2d5cfef None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:11:00.482592 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 206/409] 10.222.0.15 () {56 vars in 1180 bytes} [Thu Sep 15 12:11:00 2022] GET /identity/v3/groups/a53b135c5f244755bca9b0669c51acd8 => generated 226 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:11:00.490718 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1e4787-d05b-442a-9857-caf0b2d5cfef None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:00.604672 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1e4787-d05b-442a-9857-caf0b2d5cfef None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:00.619948 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:11:00.644068 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e995220c-d211-43c7-81bf-0c6100d69bca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:00.644531 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3561cabe-7331-472e-9985-c224834a5986 None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:00.645303 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e995220c-d211-43c7-81bf-0c6100d69bca None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:00.645823 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e995220c-d211-43c7-81bf-0c6100d69bca None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:00.646352 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e995220c-d211-43c7-81bf-0c6100d69bca None admin] PATH_INFO: `/v3/projects/bd9c06e0f78e4d26902fd521a2fe013e` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:00.648872 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e995220c-d211-43c7-81bf-0c6100d69bca None admin] RBAC: Authorizing `identity:get_project(project_id=bd9c06e0f78e4d26902fd521a2fe013e)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:11:00.650523 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e995220c-d211-43c7-81bf-0c6100d69bca None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:11:00.652878 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 207/410] 10.222.0.15 () {56 vars in 1186 bytes} [Thu Sep 15 12:11:00 2022] GET /identity/v3/projects/bd9c06e0f78e4d26902fd521a2fe013e => generated 302 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:11:00.654238 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab6eb03-965c-4013-84ab-628787b0cd80 None None] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:00.654798 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab6eb03-965c-4013-84ab-628787b0cd80 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:00.655260 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab6eb03-965c-4013-84ab-628787b0cd80 None None] PATH_INFO: `/` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:00.656770 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 208/411] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 12:11:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:11:00.659026 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab6eb03-965c-4013-84ab-628787b0cd80 None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:00.684665 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f11d85e-6777-4de3-8cea-f46c7fd88ceb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:00.685398 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f11d85e-6777-4de3-8cea-f46c7fd88ceb None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:00.685652 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f11d85e-6777-4de3-8cea-f46c7fd88ceb None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:00.685980 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f11d85e-6777-4de3-8cea-f46c7fd88ceb None admin] PATH_INFO: `/v3/roles` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:00.686648 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f11d85e-6777-4de3-8cea-f46c7fd88ceb None admin] RBAC: Adding query filter params (name=admin) {{(pid=81449) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:11:00.686949 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f11d85e-6777-4de3-8cea-f46c7fd88ceb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:11:00.688582 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f11d85e-6777-4de3-8cea-f46c7fd88ceb None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:11:00.700802 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 204/412] 10.222.0.15 () {56 vars in 1099 bytes} [Thu Sep 15 12:11:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 12:11:00.700802 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff49e9d0-9aa0-4a76-ba83-af3e3255ccb4 None None] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:00.700802 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff49e9d0-9aa0-4a76-ba83-af3e3255ccb4 None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:00.700802 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff49e9d0-9aa0-4a76-ba83-af3e3255ccb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:00.719928 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.auth.core [None req-ff49e9d0-9aa0-4a76-ba83-af3e3255ccb4 None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81449) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:11:00.721195 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-ff49e9d0-9aa0-4a76-ba83-af3e3255ccb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:11:00.733320 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:11:00.785523 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ab6eb03-965c-4013-84ab-628787b0cd80 None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:00.790757 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 205/413] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 12:11:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 12:11:00.792735 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff49e9d0-9aa0-4a76-ba83-af3e3255ccb4 None None] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:00.825971 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b64e30-9cec-4854-b226-df4b57439808 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:00.827925 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b64e30-9cec-4854-b226-df4b57439808 None admin] REQUEST_METHOD: `PUT` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:00.833142 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b64e30-9cec-4854-b226-df4b57439808 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:00.833784 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b64e30-9cec-4854-b226-df4b57439808 None admin] PATH_INFO: `/v3/projects/bd9c06e0f78e4d26902fd521a2fe013e/groups/a53b135c5f244755bca9b0669c51acd8/roles/1081b93ddfd547649ba8582f7a6e719e` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:00.842008 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b64e30-9cec-4854-b226-df4b57439808 None admin] RBAC: Authorizing `identity:create_grant(project_id=bd9c06e0f78e4d26902fd521a2fe013e, group_id=a53b135c5f244755bca9b0669c51acd8, role_id=1081b93ddfd547649ba8582f7a6e719e)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:11:00.843447 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b64e30-9cec-4854-b226-df4b57439808 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:11:00.883629 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 209/414] 10.222.0.15 () {58 vars in 1442 bytes} [Thu Sep 15 12:11:00 2022] PUT /identity/v3/projects/bd9c06e0f78e4d26902fd521a2fe013e/groups/a53b135c5f244755bca9b0669c51acd8/roles/1081b93ddfd547649ba8582f7a6e719e => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 12:11:00.885401 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:11:00.925876 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff49e9d0-9aa0-4a76-ba83-af3e3255ccb4 None None] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:00.972583 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfe9e4a-6d47-4747-ba65-5ca5e1e81d2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:00.973643 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfe9e4a-6d47-4747-ba65-5ca5e1e81d2f None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:00.974129 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfe9e4a-6d47-4747-ba65-5ca5e1e81d2f None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:00.974635 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfe9e4a-6d47-4747-ba65-5ca5e1e81d2f None admin] PATH_INFO: `/v3/groups/808ddc3a7fe547139412ee50b7269ea7` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:00.978619 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bfe9e4a-6d47-4747-ba65-5ca5e1e81d2f None admin] RBAC: Authorizing `identity:get_group(group_id=808ddc3a7fe547139412ee50b7269ea7)` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:11:00.980418 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bfe9e4a-6d47-4747-ba65-5ca5e1e81d2f None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:11:00.982164 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 206/415] 10.222.0.15 () {56 vars in 1180 bytes} [Thu Sep 15 12:11:00 2022] GET /identity/v3/groups/808ddc3a7fe547139412ee50b7269ea7 => generated 229 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:11:00.988178 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b64e30-9cec-4854-b226-df4b57439808 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:01.103713 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b64e30-9cec-4854-b226-df4b57439808 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:01.121275 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:11:01.143149 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f586f33-87eb-4f58-a9fc-3d1030b29c5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:01.143854 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f586f33-87eb-4f58-a9fc-3d1030b29c5a None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:01.144129 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f586f33-87eb-4f58-a9fc-3d1030b29c5a None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:01.144396 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f586f33-87eb-4f58-a9fc-3d1030b29c5a None admin] PATH_INFO: `/v3/projects/69bea7b802b04b689af0d31b020f8554` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:01.145583 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f586f33-87eb-4f58-a9fc-3d1030b29c5a None admin] RBAC: Authorizing `identity:get_project(project_id=69bea7b802b04b689af0d31b020f8554)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:11:01.147113 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f586f33-87eb-4f58-a9fc-3d1030b29c5a None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:11:01.149030 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 210/416] 10.222.0.15 () {56 vars in 1186 bytes} [Thu Sep 15 12:11:00 2022] GET /identity/v3/projects/69bea7b802b04b689af0d31b020f8554 => generated 344 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:11:01.154701 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfe9e4a-6d47-4747-ba65-5ca5e1e81d2f None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:01.162324 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f586f33-87eb-4f58-a9fc-3d1030b29c5a None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:01.298304 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f586f33-87eb-4f58-a9fc-3d1030b29c5a None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:01.308526 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfe9e4a-6d47-4747-ba65-5ca5e1e81d2f None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:01.363539 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ced8f3-0950-4819-bffd-7786f05ebbfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:01.364185 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ced8f3-0950-4819-bffd-7786f05ebbfe None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:01.364459 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ced8f3-0950-4819-bffd-7786f05ebbfe None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:01.364725 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ced8f3-0950-4819-bffd-7786f05ebbfe None admin] PATH_INFO: `/v3/roles/member` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:01.374983 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634f611f-874f-4a91-812f-52dd3518ba6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:01.375765 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634f611f-874f-4a91-812f-52dd3518ba6e None admin] REQUEST_METHOD: `PUT` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:01.376043 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634f611f-874f-4a91-812f-52dd3518ba6e None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:01.376311 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634f611f-874f-4a91-812f-52dd3518ba6e None admin] PATH_INFO: `/v3/projects/69bea7b802b04b689af0d31b020f8554/groups/808ddc3a7fe547139412ee50b7269ea7/roles/9c769b5c82884766b2c510b1f0f1364c` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:01.379192 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634f611f-874f-4a91-812f-52dd3518ba6e None admin] RBAC: Authorizing `identity:create_grant(project_id=69bea7b802b04b689af0d31b020f8554, group_id=808ddc3a7fe547139412ee50b7269ea7, role_id=9c769b5c82884766b2c510b1f0f1364c)` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:11:01.380625 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634f611f-874f-4a91-812f-52dd3518ba6e None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:11:01.381939 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ced8f3-0950-4819-bffd-7786f05ebbfe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 12:11:01.383839 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:11:01.391533 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application [None req-39ced8f3-0950-4819-bffd-7786f05ebbfe None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 15 12:11:01.391533 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:11:01.391533 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:11:01.391533 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:11:01.391533 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:11:01.391533 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:11:01.391533 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:11:01.391533 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:11:01.391533 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:11:01.391533 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:11:01.391533 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:11:01.391533 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:11:01.391533 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 12:11:01.391533 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 12:11:01.391533 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 12:11:01.391533 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 12:11:01.392633 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 12:11:01.392633 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 12:11:01.392633 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 12:11:01.392633 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 12:11:01.392633 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:11:01.392633 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:11:01.392633 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:11:01.392633 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:11:01.392633 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:11:01.392633 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:11:01.392633 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:11:01.392633 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application with Lock( Sep 15 12:11:01.392633 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:11:01.392633 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:11:01.392633 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:11:01.392633 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:11:01.393464 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:11:01.393464 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:11:01.393464 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:11:01.393464 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:11:01.393464 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 12:11:01.393464 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 12:11:01.393464 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 12:11:01.393464 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 12:11:01.393464 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 12:11:01.393464 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 12:11:01.393464 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 15 12:11:01.393464 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 15 12:11:01.397666 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 211/417] 10.222.0.15 () {56 vars in 1099 bytes} [Thu Sep 15 12:11:01 2022] GET /identity/v3/roles/member => generated 84 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 15 12:11:01.399500 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ced8f3-0950-4819-bffd-7786f05ebbfe None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:01.401548 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 207/418] 10.222.0.15 () {58 vars in 1442 bytes} [Thu Sep 15 12:11:01 2022] PUT /identity/v3/projects/69bea7b802b04b689af0d31b020f8554/groups/808ddc3a7fe547139412ee50b7269ea7/roles/9c769b5c82884766b2c510b1f0f1364c => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:11:01.464236 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:11:01.540933 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ced8f3-0950-4819-bffd-7786f05ebbfe None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:01.591698 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cec8aab-c37b-4586-8a96-7c68390db1c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:01.592662 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cec8aab-c37b-4586-8a96-7c68390db1c0 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:01.593181 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cec8aab-c37b-4586-8a96-7c68390db1c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:01.593649 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cec8aab-c37b-4586-8a96-7c68390db1c0 None admin] PATH_INFO: `/v3/roles` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:01.594504 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cec8aab-c37b-4586-8a96-7c68390db1c0 None admin] RBAC: Adding query filter params (name=member) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:11:01.594987 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cec8aab-c37b-4586-8a96-7c68390db1c0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:11:01.596542 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cec8aab-c37b-4586-8a96-7c68390db1c0 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:11:01.606946 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 212/419] 10.222.0.15 () {56 vars in 1101 bytes} [Thu Sep 15 12:11:01 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:11:01.612421 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634f611f-874f-4a91-812f-52dd3518ba6e None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:01.669847 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45232992-26c3-4864-b92b-22a29e001f14 None None] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:01.670167 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45232992-26c3-4864-b92b-22a29e001f14 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:01.670379 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45232992-26c3-4864-b92b-22a29e001f14 None None] PATH_INFO: `/` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:01.671820 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 213/420] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 12:11:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:11:01.677143 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef98067-5be6-4a4a-bc74-167d846dc7a0 None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:01.677431 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef98067-5be6-4a4a-bc74-167d846dc7a0 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:01.677737 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef98067-5be6-4a4a-bc74-167d846dc7a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:01.702797 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-5ef98067-5be6-4a4a-bc74-167d846dc7a0 None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:11:01.705025 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-5ef98067-5be6-4a4a-bc74-167d846dc7a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:11:01.737058 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634f611f-874f-4a91-812f-52dd3518ba6e None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:01.786009 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fcc5a67-ae74-40dd-9ad0-351baad82353 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:01.786969 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcc5a67-ae74-40dd-9ad0-351baad82353 None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:01.787985 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcc5a67-ae74-40dd-9ad0-351baad82353 None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:01.788392 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fcc5a67-ae74-40dd-9ad0-351baad82353 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:01.803652 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 214/421] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 12:11:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 12:11:01.882252 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:11:01.889744 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fcc5a67-ae74-40dd-9ad0-351baad82353 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:11:01.891467 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fcc5a67-ae74-40dd-9ad0-351baad82353 None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:11:02.024966 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 208/422] 10.222.0.15 () {58 vars in 1282 bytes} [Thu Sep 15 12:11:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 413 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 15 12:11:02.033830 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef98067-5be6-4a4a-bc74-167d846dc7a0 None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:02.085613 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:11:02.174382 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef98067-5be6-4a4a-bc74-167d846dc7a0 None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:02.223229 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7b7c5d-27f5-4356-acc7-3a482ea58e73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:02.223876 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7b7c5d-27f5-4356-acc7-3a482ea58e73 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:02.224151 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7b7c5d-27f5-4356-acc7-3a482ea58e73 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:02.224420 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7b7c5d-27f5-4356-acc7-3a482ea58e73 None admin] PATH_INFO: `/v3/projects/10a21ffb0b8d4800a805c76f171c40c1` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:02.226110 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7b7c5d-27f5-4356-acc7-3a482ea58e73 None admin] RBAC: Authorizing `identity:get_project(project_id=10a21ffb0b8d4800a805c76f171c40c1)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:11:02.227533 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7b7c5d-27f5-4356-acc7-3a482ea58e73 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:11:02.229160 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 215/423] 10.222.0.15 () {56 vars in 1186 bytes} [Thu Sep 15 12:11:02 2022] GET /identity/v3/projects/10a21ffb0b8d4800a805c76f171c40c1 => generated 298 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:11:02.235085 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fcc5a67-ae74-40dd-9ad0-351baad82353 None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:02.258888 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7b7c5d-27f5-4356-acc7-3a482ea58e73 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:02.420523 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:11:02.424888 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7b7c5d-27f5-4356-acc7-3a482ea58e73 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:02.429962 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:11:02.431984 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fcc5a67-ae74-40dd-9ad0-351baad82353 None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:02.481325 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da755306-f6a2-4cd1-b939-641c2dfa54de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:02.481533 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14870b3e-cec1-495c-aea8-523fee05acaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:02.481990 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da755306-f6a2-4cd1-b939-641c2dfa54de None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:02.482231 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14870b3e-cec1-495c-aea8-523fee05acaf None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:02.482335 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da755306-f6a2-4cd1-b939-641c2dfa54de None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:02.482407 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14870b3e-cec1-495c-aea8-523fee05acaf None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:02.482472 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da755306-f6a2-4cd1-b939-641c2dfa54de None admin] PATH_INFO: `/v3/groups/a53b135c5f244755bca9b0669c51acd8` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:02.482682 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14870b3e-cec1-495c-aea8-523fee05acaf None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:02.483605 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da755306-f6a2-4cd1-b939-641c2dfa54de None admin] RBAC: Authorizing `identity:get_group(group_id=a53b135c5f244755bca9b0669c51acd8)` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:11:02.485353 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da755306-f6a2-4cd1-b939-641c2dfa54de None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:11:02.487782 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 209/424] 10.222.0.15 () {56 vars in 1180 bytes} [Thu Sep 15 12:11:02 2022] GET /identity/v3/groups/a53b135c5f244755bca9b0669c51acd8 => generated 226 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:11:02.489419 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c0966c-f400-4d87-a109-977d8b986cfe None None] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:02.489705 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c0966c-f400-4d87-a109-977d8b986cfe None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:02.489967 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c0966c-f400-4d87-a109-977d8b986cfe None None] PATH_INFO: `/` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:02.491821 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 210/425] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 12:11:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:11:02.493735 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c0966c-f400-4d87-a109-977d8b986cfe None None] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:02.499995 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14870b3e-cec1-495c-aea8-523fee05acaf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 12:11:02.504870 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application [None req-14870b3e-cec1-495c-aea8-523fee05acaf None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 15 12:11:02.504870 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:11:02.504870 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:11:02.504870 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:11:02.504870 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:11:02.504870 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:11:02.504870 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:11:02.504870 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:11:02.504870 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:11:02.504870 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:11:02.504870 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:11:02.504870 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:11:02.504870 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 12:11:02.504870 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 12:11:02.504870 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 12:11:02.504870 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 12:11:02.505968 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 12:11:02.505968 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 12:11:02.505968 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 12:11:02.505968 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 12:11:02.505968 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:11:02.505968 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:11:02.505968 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:11:02.505968 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:11:02.505968 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:11:02.505968 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:11:02.505968 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:11:02.505968 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application with Lock( Sep 15 12:11:02.505968 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:11:02.505968 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:11:02.505968 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:11:02.505968 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:11:02.506971 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:11:02.506971 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:11:02.506971 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:11:02.506971 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:11:02.506971 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 12:11:02.506971 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 12:11:02.506971 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 12:11:02.506971 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 12:11:02.506971 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 12:11:02.506971 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 12:11:02.506971 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 15 12:11:02.506971 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 15 12:11:02.506971 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 216/426] 10.222.0.15 () {56 vars in 1114 bytes} [Thu Sep 15 12:11:02 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:11:02.507635 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8e8719-5173-4416-a937-8eceeb9b056c None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:02.507831 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8e8719-5173-4416-a937-8eceeb9b056c None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:02.508620 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8e8719-5173-4416-a937-8eceeb9b056c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:02.528415 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-2a8e8719-5173-4416-a937-8eceeb9b056c None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:11:02.529663 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-2a8e8719-5173-4416-a937-8eceeb9b056c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:11:02.612864 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 217/427] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 12:11:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 12:11:02.613767 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8e8719-5173-4416-a937-8eceeb9b056c None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:02.644882 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c0966c-f400-4d87-a109-977d8b986cfe None None] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:02.709780 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf329e8-ae03-412d-bc9a-8907cab69286 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:02.710393 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf329e8-ae03-412d-bc9a-8907cab69286 None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:02.710671 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf329e8-ae03-412d-bc9a-8907cab69286 None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:02.710941 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf329e8-ae03-412d-bc9a-8907cab69286 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:02.711655 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf329e8-ae03-412d-bc9a-8907cab69286 None admin] RBAC: Adding query filter params (group.id=a53b135c5f244755bca9b0669c51acd8, role.id=0679d6628a25438d90dc4bdc0401eda2, scope.project.id=10a21ffb0b8d4800a805c76f171c40c1) {{(pid=81449) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:11:02.711960 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf329e8-ae03-412d-bc9a-8907cab69286 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:11:02.713420 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf329e8-ae03-412d-bc9a-8907cab69286 None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:11:02.726073 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 211/428] 10.222.0.15 () {56 vars in 1376 bytes} [Thu Sep 15 12:11:02 2022] GET /identity/v3/role_assignments?group.id=a53b135c5f244755bca9b0669c51acd8&scope.project.id=10a21ffb0b8d4800a805c76f171c40c1&role.id=0679d6628a25438d90dc4bdc0401eda2 => generated 611 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:11:02.763995 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:11:02.772361 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8e8719-5173-4416-a937-8eceeb9b056c None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:02.809044 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a027f618-a714-4093-9d81-bdc233146b7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:02.809657 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a027f618-a714-4093-9d81-bdc233146b7b None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:02.809934 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a027f618-a714-4093-9d81-bdc233146b7b None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:02.810203 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a027f618-a714-4093-9d81-bdc233146b7b None admin] PATH_INFO: `/v3/roles` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:02.810868 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a027f618-a714-4093-9d81-bdc233146b7b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:11:02.811167 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a027f618-a714-4093-9d81-bdc233146b7b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:11:02.812524 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a027f618-a714-4093-9d81-bdc233146b7b None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:11:02.822062 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 218/429] 10.222.0.15 () {56 vars in 1111 bytes} [Thu Sep 15 12:11:02 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:11:02.830172 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf329e8-ae03-412d-bc9a-8907cab69286 None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:02.873894 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:11:02.992143 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf329e8-ae03-412d-bc9a-8907cab69286 None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:02.994774 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a027f618-a714-4093-9d81-bdc233146b7b None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:03.039502 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865ddd31-4180-4e76-9640-5af5dcc69e33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:03.040115 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865ddd31-4180-4e76-9640-5af5dcc69e33 None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:03.040393 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865ddd31-4180-4e76-9640-5af5dcc69e33 None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:03.040663 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865ddd31-4180-4e76-9640-5af5dcc69e33 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:03.127874 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-865ddd31-4180-4e76-9640-5af5dcc69e33 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:11:03.129580 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-865ddd31-4180-4e76-9640-5af5dcc69e33 None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:11:03.138313 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a027f618-a714-4093-9d81-bdc233146b7b None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:03.171560 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:11:03.191747 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7fbabcd-7d42-4a45-83a9-54ac375fa20b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:03.192676 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fbabcd-7d42-4a45-83a9-54ac375fa20b None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:03.192994 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fbabcd-7d42-4a45-83a9-54ac375fa20b None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:03.193287 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fbabcd-7d42-4a45-83a9-54ac375fa20b None admin] PATH_INFO: `/v3/roles/member` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:03.212992 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7fbabcd-7d42-4a45-83a9-54ac375fa20b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 12:11:03.220944 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application [None req-c7fbabcd-7d42-4a45-83a9-54ac375fa20b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Sep 15 12:11:03.220944 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:11:03.220944 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:11:03.220944 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:11:03.220944 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:11:03.220944 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:11:03.220944 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:11:03.220944 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:11:03.220944 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:11:03.220944 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:11:03.220944 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:11:03.220944 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:11:03.220944 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 12:11:03.220944 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 12:11:03.220944 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 12:11:03.220944 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 12:11:03.223379 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 12:11:03.223379 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 12:11:03.223379 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 12:11:03.223379 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 12:11:03.223379 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:11:03.223379 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:11:03.223379 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:11:03.223379 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:11:03.223379 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:11:03.223379 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:11:03.223379 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:11:03.223379 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application with Lock( Sep 15 12:11:03.223379 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:11:03.223379 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:11:03.223379 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:11:03.223379 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:11:03.233234 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:11:03.233234 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:11:03.233234 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:11:03.233234 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:11:03.233234 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 12:11:03.233234 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 12:11:03.233234 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 12:11:03.233234 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 12:11:03.233234 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 12:11:03.233234 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 12:11:03.233234 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Sep 15 12:11:03.233234 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 15 12:11:03.233234 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 219/430] 10.222.0.15 () {56 vars in 1099 bytes} [Thu Sep 15 12:11:02 2022] GET /identity/v3/roles/member => generated 84 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 15 12:11:03.233234 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7fbabcd-7d42-4a45-83a9-54ac375fa20b None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:03.253274 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:11:03.275819 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 212/431] 10.222.0.15 () {58 vars in 1282 bytes} [Thu Sep 15 12:11:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 446 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 15 12:11:03.282122 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865ddd31-4180-4e76-9640-5af5dcc69e33 None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:03.362363 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7fbabcd-7d42-4a45-83a9-54ac375fa20b None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:03.405523 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865ddd31-4180-4e76-9640-5af5dcc69e33 None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:03.406767 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de18cbb-833b-41dc-b42c-28629b7437f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:03.407572 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de18cbb-833b-41dc-b42c-28629b7437f5 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:03.408718 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de18cbb-833b-41dc-b42c-28629b7437f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:03.409032 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de18cbb-833b-41dc-b42c-28629b7437f5 None admin] PATH_INFO: `/v3/roles` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:03.409714 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de18cbb-833b-41dc-b42c-28629b7437f5 None admin] RBAC: Adding query filter params (name=member) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:11:03.410058 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de18cbb-833b-41dc-b42c-28629b7437f5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:11:03.411483 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de18cbb-833b-41dc-b42c-28629b7437f5 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:11:03.423442 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 220/432] 10.222.0.15 () {56 vars in 1101 bytes} [Thu Sep 15 12:11:03 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 12:11:03.429233 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de18cbb-833b-41dc-b42c-28629b7437f5 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:03.436169 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:11:03.445941 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031840fc-a799-4362-bb5c-fba3166e8ebc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:03.446581 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031840fc-a799-4362-bb5c-fba3166e8ebc None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:03.446860 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031840fc-a799-4362-bb5c-fba3166e8ebc None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:03.447129 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031840fc-a799-4362-bb5c-fba3166e8ebc None admin] PATH_INFO: `/v3/projects/10a21ffb0b8d4800a805c76f171c40c1` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:03.448259 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031840fc-a799-4362-bb5c-fba3166e8ebc None admin] RBAC: Authorizing `identity:get_project(project_id=10a21ffb0b8d4800a805c76f171c40c1)` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:11:03.449750 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031840fc-a799-4362-bb5c-fba3166e8ebc None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:11:03.451678 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 213/433] 10.222.0.15 () {56 vars in 1186 bytes} [Thu Sep 15 12:11:03 2022] GET /identity/v3/projects/10a21ffb0b8d4800a805c76f171c40c1 => generated 298 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:11:03.456616 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031840fc-a799-4362-bb5c-fba3166e8ebc None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:03.524232 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:11:03.557592 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de18cbb-833b-41dc-b42c-28629b7437f5 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:03.582937 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031840fc-a799-4362-bb5c-fba3166e8ebc None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:03.599040 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d54e74-113e-4447-95d4-9ea7380712b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:03.599697 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d54e74-113e-4447-95d4-9ea7380712b5 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:03.599982 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d54e74-113e-4447-95d4-9ea7380712b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:03.600254 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d54e74-113e-4447-95d4-9ea7380712b5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:03.622798 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057b1318-7013-4ae6-a6b5-96b27afdec8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:03.623626 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057b1318-7013-4ae6-a6b5-96b27afdec8c None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:03.623911 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057b1318-7013-4ae6-a6b5-96b27afdec8c None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:03.624182 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057b1318-7013-4ae6-a6b5-96b27afdec8c None admin] PATH_INFO: `/v3/groups/a53b135c5f244755bca9b0669c51acd8` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:03.625335 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057b1318-7013-4ae6-a6b5-96b27afdec8c None admin] RBAC: Authorizing `identity:get_group(group_id=a53b135c5f244755bca9b0669c51acd8)` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:11:03.626946 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057b1318-7013-4ae6-a6b5-96b27afdec8c None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:11:03.628824 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 214/434] 10.222.0.15 () {56 vars in 1180 bytes} [Thu Sep 15 12:11:03 2022] GET /identity/v3/groups/a53b135c5f244755bca9b0669c51acd8 => generated 226 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 12:11:03.635148 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057b1318-7013-4ae6-a6b5-96b27afdec8c None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:03.697493 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d54e74-113e-4447-95d4-9ea7380712b5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:11:03.698951 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d54e74-113e-4447-95d4-9ea7380712b5 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:11:03.718473 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:11:03.763476 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057b1318-7013-4ae6-a6b5-96b27afdec8c None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:03.791042 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:11:03.800191 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c711b1db-66ef-4193-b902-2dc807ec59b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:03.800835 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c711b1db-66ef-4193-b902-2dc807ec59b6 None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:03.801118 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c711b1db-66ef-4193-b902-2dc807ec59b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:03.801389 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c711b1db-66ef-4193-b902-2dc807ec59b6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:03.802114 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c711b1db-66ef-4193-b902-2dc807ec59b6 None admin] RBAC: Adding query filter params (group.id=a53b135c5f244755bca9b0669c51acd8, role.id=1081b93ddfd547649ba8582f7a6e719e, scope.project.id=10a21ffb0b8d4800a805c76f171c40c1) {{(pid=81449) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:11:03.802424 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c711b1db-66ef-4193-b902-2dc807ec59b6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:11:03.803946 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c711b1db-66ef-4193-b902-2dc807ec59b6 None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:11:03.809556 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 215/435] 10.222.0.15 () {56 vars in 1376 bytes} [Thu Sep 15 12:11:03 2022] GET /identity/v3/role_assignments?group.id=a53b135c5f244755bca9b0669c51acd8&scope.project.id=10a21ffb0b8d4800a805c76f171c40c1&role.id=1081b93ddfd547649ba8582f7a6e719e => generated 611 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:11:03.810827 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863b3fd7-7e71-4fc3-b365-dd8c6fdb736e None None] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:03.811114 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863b3fd7-7e71-4fc3-b365-dd8c6fdb736e None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:03.811435 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863b3fd7-7e71-4fc3-b365-dd8c6fdb736e None None] PATH_INFO: `/` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:03.812904 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 216/436] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 12:11:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:11:03.819123 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b243a5da-8ba9-4e35-92d7-ea1a140a56e0 None None] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:03.819411 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b243a5da-8ba9-4e35-92d7-ea1a140a56e0 None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:03.819685 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b243a5da-8ba9-4e35-92d7-ea1a140a56e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:03.826105 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 221/437] 10.222.0.15 () {58 vars in 1282 bytes} [Thu Sep 15 12:11:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 15 12:11:03.833918 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d54e74-113e-4447-95d4-9ea7380712b5 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:03.843157 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.auth.core [None req-b243a5da-8ba9-4e35-92d7-ea1a140a56e0 None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81449) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:11:03.844369 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-b243a5da-8ba9-4e35-92d7-ea1a140a56e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:11:03.920807 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 217/438] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 12:11:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 15 12:11:03.921041 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5873e648-9ad8-42fb-8a1d-28b0b0d2c2c5 None None] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:03.921319 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5873e648-9ad8-42fb-8a1d-28b0b0d2c2c5 None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:03.921584 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5873e648-9ad8-42fb-8a1d-28b0b0d2c2c5 None None] PATH_INFO: `/` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:03.922982 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 218/439] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 12:11:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:11:03.928929 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105162c0-269f-4d86-aaed-beb46ad6fa26 None None] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:03.929212 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105162c0-269f-4d86-aaed-beb46ad6fa26 None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:03.929470 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105162c0-269f-4d86-aaed-beb46ad6fa26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:03.948996 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.auth.core [None req-105162c0-269f-4d86-aaed-beb46ad6fa26 None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81449) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:11:03.950286 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-105162c0-269f-4d86-aaed-beb46ad6fa26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:11:03.965937 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d54e74-113e-4447-95d4-9ea7380712b5 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:04.009248 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4414914-c447-411a-8d18-176a7051310d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:04.009939 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4414914-c447-411a-8d18-176a7051310d None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:04.010220 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4414914-c447-411a-8d18-176a7051310d None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:04.010494 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4414914-c447-411a-8d18-176a7051310d None admin] PATH_INFO: `/v3/projects/bd9c06e0f78e4d26902fd521a2fe013e` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:04.011613 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4414914-c447-411a-8d18-176a7051310d None admin] RBAC: Authorizing `identity:get_project(project_id=bd9c06e0f78e4d26902fd521a2fe013e)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:11:04.014173 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4414914-c447-411a-8d18-176a7051310d None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:11:04.015096 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 222/440] 10.222.0.15 () {56 vars in 1186 bytes} [Thu Sep 15 12:11:03 2022] GET /identity/v3/projects/bd9c06e0f78e4d26902fd521a2fe013e => generated 302 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:11:04.020373 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4414914-c447-411a-8d18-176a7051310d None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:04.029121 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:11:04.046774 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 219/441] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 12:11:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 15 12:11:04.076503 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:11:04.133721 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4414914-c447-411a-8d18-176a7051310d None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:04.170664 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271a8938-7af5-446a-bc3a-c51c24160e4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:04.171292 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271a8938-7af5-446a-bc3a-c51c24160e4a None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:04.171567 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271a8938-7af5-446a-bc3a-c51c24160e4a None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:04.171836 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271a8938-7af5-446a-bc3a-c51c24160e4a None admin] PATH_INFO: `/v3/groups/a53b135c5f244755bca9b0669c51acd8` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:04.172999 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-271a8938-7af5-446a-bc3a-c51c24160e4a None admin] RBAC: Authorizing `identity:get_group(group_id=a53b135c5f244755bca9b0669c51acd8)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:11:04.174416 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-271a8938-7af5-446a-bc3a-c51c24160e4a None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:11:04.176118 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 223/442] 10.222.0.15 () {56 vars in 1180 bytes} [Thu Sep 15 12:11:04 2022] GET /identity/v3/groups/a53b135c5f244755bca9b0669c51acd8 => generated 226 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 12:11:04.182433 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105162c0-269f-4d86-aaed-beb46ad6fa26 None None] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:04.212039 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271a8938-7af5-446a-bc3a-c51c24160e4a None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:04.307620 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105162c0-269f-4d86-aaed-beb46ad6fa26 None None] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:04.326787 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271a8938-7af5-446a-bc3a-c51c24160e4a None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:04.340261 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:11:04.352971 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4138a063-33d0-4b44-b001-59282d651105 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:04.353607 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4138a063-33d0-4b44-b001-59282d651105 None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:04.353885 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4138a063-33d0-4b44-b001-59282d651105 None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:04.354155 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4138a063-33d0-4b44-b001-59282d651105 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:04.355129 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4138a063-33d0-4b44-b001-59282d651105 None admin] RBAC: Adding query filter params (group.id=a53b135c5f244755bca9b0669c51acd8, role.id=0679d6628a25438d90dc4bdc0401eda2, scope.project.id=bd9c06e0f78e4d26902fd521a2fe013e) {{(pid=81449) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:11:04.355599 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4138a063-33d0-4b44-b001-59282d651105 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:11:04.357605 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4138a063-33d0-4b44-b001-59282d651105 None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:11:04.363649 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 220/443] 10.222.0.15 () {56 vars in 1376 bytes} [Thu Sep 15 12:11:04 2022] GET /identity/v3/role_assignments?group.id=a53b135c5f244755bca9b0669c51acd8&scope.project.id=bd9c06e0f78e4d26902fd521a2fe013e&role.id=0679d6628a25438d90dc4bdc0401eda2 => generated 611 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:11:04.365071 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4138a063-33d0-4b44-b001-59282d651105 None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:04.367876 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eda965d-da1d-49c7-a76e-c3cf8eec47c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:04.368536 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eda965d-da1d-49c7-a76e-c3cf8eec47c8 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:04.368841 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eda965d-da1d-49c7-a76e-c3cf8eec47c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:04.369242 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eda965d-da1d-49c7-a76e-c3cf8eec47c8 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:04.381220 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eda965d-da1d-49c7-a76e-c3cf8eec47c8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 12:11:04.386265 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application [None req-1eda965d-da1d-49c7-a76e-c3cf8eec47c8 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 15 12:11:04.386265 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:11:04.386265 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:11:04.386265 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:11:04.386265 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:11:04.386265 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:11:04.386265 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:11:04.386265 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:11:04.386265 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:11:04.386265 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:11:04.386265 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:11:04.386265 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:11:04.386265 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 12:11:04.386265 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 12:11:04.386265 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 12:11:04.386265 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 12:11:04.387254 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 12:11:04.387254 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 12:11:04.387254 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 12:11:04.387254 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 12:11:04.387254 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:11:04.387254 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:11:04.387254 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:11:04.387254 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:11:04.387254 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:11:04.387254 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:11:04.387254 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:11:04.387254 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application with Lock( Sep 15 12:11:04.387254 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:11:04.387254 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:11:04.387254 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:11:04.387254 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:11:04.388043 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:11:04.388043 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:11:04.388043 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:11:04.388043 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:11:04.388043 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 12:11:04.388043 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 12:11:04.388043 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 12:11:04.388043 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 12:11:04.388043 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 12:11:04.388043 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 12:11:04.388043 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 15 12:11:04.388043 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 15 12:11:04.388043 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 224/444] 10.222.0.15 () {56 vars in 1114 bytes} [Thu Sep 15 12:11:04 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 15 12:11:04.393517 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eda965d-da1d-49c7-a76e-c3cf8eec47c8 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:04.428583 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:11:04.478360 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4138a063-33d0-4b44-b001-59282d651105 None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:04.514763 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff31a28-3811-4504-b192-94d83cbb18b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:04.515503 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff31a28-3811-4504-b192-94d83cbb18b2 None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:04.515786 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff31a28-3811-4504-b192-94d83cbb18b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:04.516057 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff31a28-3811-4504-b192-94d83cbb18b2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:04.519684 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eda965d-da1d-49c7-a76e-c3cf8eec47c8 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:04.527264 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff31a28-3811-4504-b192-94d83cbb18b2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 12:11:04.532202 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application [None req-0ff31a28-3811-4504-b192-94d83cbb18b2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 15 12:11:04.532202 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:11:04.532202 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:11:04.532202 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:11:04.532202 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:11:04.532202 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:11:04.532202 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:11:04.532202 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:11:04.532202 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:11:04.532202 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:11:04.532202 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:11:04.532202 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:11:04.532202 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 12:11:04.532202 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 12:11:04.532202 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 12:11:04.532202 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 12:11:04.533132 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 12:11:04.533132 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 12:11:04.533132 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 12:11:04.533132 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 12:11:04.533132 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:11:04.533132 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:11:04.533132 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:11:04.533132 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:11:04.533132 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:11:04.533132 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:11:04.533132 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:11:04.533132 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application with Lock( Sep 15 12:11:04.533132 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:11:04.533132 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:11:04.533132 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:11:04.533132 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:11:04.533950 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:11:04.533950 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:11:04.533950 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:11:04.533950 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:11:04.533950 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 12:11:04.533950 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 12:11:04.533950 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 12:11:04.533950 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 12:11:04.533950 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 12:11:04.533950 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 12:11:04.533950 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 15 12:11:04.533950 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application Sep 15 12:11:04.533950 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 221/445] 10.222.0.15 () {56 vars in 1096 bytes} [Thu Sep 15 12:11:04 2022] GET /identity/v3/roles/admin => generated 83 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:11:04.539495 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff31a28-3811-4504-b192-94d83cbb18b2 None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:04.568059 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf97073d-a9bb-43a7-9ea8-df7f5b0fcbcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:04.568587 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf97073d-a9bb-43a7-9ea8-df7f5b0fcbcb None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:04.568587 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf97073d-a9bb-43a7-9ea8-df7f5b0fcbcb None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:04.568721 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf97073d-a9bb-43a7-9ea8-df7f5b0fcbcb None admin] PATH_INFO: `/v3/roles` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:04.569358 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf97073d-a9bb-43a7-9ea8-df7f5b0fcbcb None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:11:04.570152 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf97073d-a9bb-43a7-9ea8-df7f5b0fcbcb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:11:04.571079 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf97073d-a9bb-43a7-9ea8-df7f5b0fcbcb None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:11:04.582296 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 225/446] 10.222.0.15 () {56 vars in 1111 bytes} [Thu Sep 15 12:11:04 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 12:11:04.589276 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf97073d-a9bb-43a7-9ea8-df7f5b0fcbcb None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:04.629918 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:11:04.670832 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff31a28-3811-4504-b192-94d83cbb18b2 None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:04.685739 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:11:04.703455 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf97073d-a9bb-43a7-9ea8-df7f5b0fcbcb None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:04.709459 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1185cfb8-f44a-4f4b-9816-6f09823189f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:04.710067 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1185cfb8-f44a-4f4b-9816-6f09823189f6 None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:04.710343 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1185cfb8-f44a-4f4b-9816-6f09823189f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:04.710610 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1185cfb8-f44a-4f4b-9816-6f09823189f6 None admin] PATH_INFO: `/v3/roles` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:04.711275 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1185cfb8-f44a-4f4b-9816-6f09823189f6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81449) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:11:04.711576 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1185cfb8-f44a-4f4b-9816-6f09823189f6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:11:04.712997 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1185cfb8-f44a-4f4b-9816-6f09823189f6 None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:11:04.722661 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 222/447] 10.222.0.15 () {56 vars in 1099 bytes} [Thu Sep 15 12:11:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 12:11:04.729037 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1185cfb8-f44a-4f4b-9816-6f09823189f6 None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:04.743642 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5dca2f-045a-4099-98a6-73f8164ce4fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:04.744313 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5dca2f-045a-4099-98a6-73f8164ce4fb None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:04.744597 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5dca2f-045a-4099-98a6-73f8164ce4fb None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:04.745039 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5dca2f-045a-4099-98a6-73f8164ce4fb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:04.814140 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5dca2f-045a-4099-98a6-73f8164ce4fb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:11:04.815596 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5dca2f-045a-4099-98a6-73f8164ce4fb None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:11:04.832897 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1185cfb8-f44a-4f4b-9816-6f09823189f6 None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:04.848042 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:11:04.865843 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966d7197-c65f-4ecd-8561-fa58cf6c26c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:04.866455 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966d7197-c65f-4ecd-8561-fa58cf6c26c7 None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:04.866731 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966d7197-c65f-4ecd-8561-fa58cf6c26c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:04.867000 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966d7197-c65f-4ecd-8561-fa58cf6c26c7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:04.908500 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:11:04.912821 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 226/448] 10.222.0.15 () {58 vars in 1282 bytes} [Thu Sep 15 12:11:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 15 12:11:04.918298 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5dca2f-045a-4099-98a6-73f8164ce4fb None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:04.934676 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-966d7197-c65f-4ecd-8561-fa58cf6c26c7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:11:04.936146 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-966d7197-c65f-4ecd-8561-fa58cf6c26c7 None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:11:05.021788 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5dca2f-045a-4099-98a6-73f8164ce4fb None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:05.041298 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 223/449] 10.222.0.15 () {58 vars in 1282 bytes} [Thu Sep 15 12:11:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 15 12:11:05.046995 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966d7197-c65f-4ecd-8561-fa58cf6c26c7 None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:05.055750 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68938731-650b-4d1a-ad5a-15f93868eeb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:05.056377 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68938731-650b-4d1a-ad5a-15f93868eeb1 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:05.056656 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68938731-650b-4d1a-ad5a-15f93868eeb1 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:05.056955 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68938731-650b-4d1a-ad5a-15f93868eeb1 None admin] PATH_INFO: `/v3/projects/bd9c06e0f78e4d26902fd521a2fe013e` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:05.058046 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68938731-650b-4d1a-ad5a-15f93868eeb1 None admin] RBAC: Authorizing `identity:get_project(project_id=bd9c06e0f78e4d26902fd521a2fe013e)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:11:05.059456 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68938731-650b-4d1a-ad5a-15f93868eeb1 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:11:05.061226 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 227/450] 10.222.0.15 () {56 vars in 1186 bytes} [Thu Sep 15 12:11:04 2022] GET /identity/v3/projects/bd9c06e0f78e4d26902fd521a2fe013e => generated 302 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 12:11:05.066264 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68938731-650b-4d1a-ad5a-15f93868eeb1 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:05.069785 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:11:05.121109 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:11:05.145488 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966d7197-c65f-4ecd-8561-fa58cf6c26c7 None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:05.163774 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68938731-650b-4d1a-ad5a-15f93868eeb1 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:05.179907 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3275bf3-bc55-4205-b35a-712ede4e5515 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:05.180537 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3275bf3-bc55-4205-b35a-712ede4e5515 None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:05.180845 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3275bf3-bc55-4205-b35a-712ede4e5515 None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:05.181120 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3275bf3-bc55-4205-b35a-712ede4e5515 None admin] PATH_INFO: `/v3/projects/69bea7b802b04b689af0d31b020f8554` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:05.182235 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3275bf3-bc55-4205-b35a-712ede4e5515 None admin] RBAC: Authorizing `identity:get_project(project_id=69bea7b802b04b689af0d31b020f8554)` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:11:05.183686 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3275bf3-bc55-4205-b35a-712ede4e5515 None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:11:05.185317 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 224/451] 10.222.0.15 () {56 vars in 1186 bytes} [Thu Sep 15 12:11:05 2022] GET /identity/v3/projects/69bea7b802b04b689af0d31b020f8554 => generated 344 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 12:11:05.190768 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3275bf3-bc55-4205-b35a-712ede4e5515 None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:05.196500 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1913a799-954e-4d08-8ed5-20d941d718d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:05.197172 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1913a799-954e-4d08-8ed5-20d941d718d1 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:05.197452 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1913a799-954e-4d08-8ed5-20d941d718d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:05.197722 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1913a799-954e-4d08-8ed5-20d941d718d1 None admin] PATH_INFO: `/v3/groups/a53b135c5f244755bca9b0669c51acd8` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:05.198825 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1913a799-954e-4d08-8ed5-20d941d718d1 None admin] RBAC: Authorizing `identity:get_group(group_id=a53b135c5f244755bca9b0669c51acd8)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:11:05.200237 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1913a799-954e-4d08-8ed5-20d941d718d1 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:11:05.206390 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 228/452] 10.222.0.15 () {56 vars in 1180 bytes} [Thu Sep 15 12:11:05 2022] GET /identity/v3/groups/a53b135c5f244755bca9b0669c51acd8 => generated 226 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 12:11:05.207259 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1913a799-954e-4d08-8ed5-20d941d718d1 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:05.279102 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:11:05.297915 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3275bf3-bc55-4205-b35a-712ede4e5515 None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:05.317034 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1913a799-954e-4d08-8ed5-20d941d718d1 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:05.331174 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c497ce-5cdb-41b6-884d-0d9cebe80057 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:05.331944 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c497ce-5cdb-41b6-884d-0d9cebe80057 None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:05.332218 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c497ce-5cdb-41b6-884d-0d9cebe80057 None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:05.332483 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c497ce-5cdb-41b6-884d-0d9cebe80057 None admin] PATH_INFO: `/v3/groups/808ddc3a7fe547139412ee50b7269ea7` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:05.333614 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c497ce-5cdb-41b6-884d-0d9cebe80057 None admin] RBAC: Authorizing `identity:get_group(group_id=808ddc3a7fe547139412ee50b7269ea7)` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:11:05.335054 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c497ce-5cdb-41b6-884d-0d9cebe80057 None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:11:05.336613 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 225/453] 10.222.0.15 () {56 vars in 1180 bytes} [Thu Sep 15 12:11:05 2022] GET /identity/v3/groups/808ddc3a7fe547139412ee50b7269ea7 => generated 229 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 12:11:05.342257 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c497ce-5cdb-41b6-884d-0d9cebe80057 None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:05.350271 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9adb80ed-c27b-40d4-a967-e927299d2770 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:05.350864 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adb80ed-c27b-40d4-a967-e927299d2770 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:05.351142 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adb80ed-c27b-40d4-a967-e927299d2770 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:05.351411 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adb80ed-c27b-40d4-a967-e927299d2770 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:05.352134 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9adb80ed-c27b-40d4-a967-e927299d2770 None admin] RBAC: Adding query filter params (group.id=a53b135c5f244755bca9b0669c51acd8, role.id=1081b93ddfd547649ba8582f7a6e719e, scope.project.id=bd9c06e0f78e4d26902fd521a2fe013e) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:11:05.352444 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9adb80ed-c27b-40d4-a967-e927299d2770 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:11:05.353890 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9adb80ed-c27b-40d4-a967-e927299d2770 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:11:05.354126 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:11:05.360226 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 229/454] 10.222.0.15 () {56 vars in 1376 bytes} [Thu Sep 15 12:11:05 2022] GET /identity/v3/role_assignments?group.id=a53b135c5f244755bca9b0669c51acd8&scope.project.id=bd9c06e0f78e4d26902fd521a2fe013e&role.id=1081b93ddfd547649ba8582f7a6e719e => generated 611 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:11:05.447693 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c497ce-5cdb-41b6-884d-0d9cebe80057 None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:05.483961 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05eb4212-635e-4acb-9d4d-375d2804952c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:05.484586 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05eb4212-635e-4acb-9d4d-375d2804952c None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:05.484899 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05eb4212-635e-4acb-9d4d-375d2804952c None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:05.485177 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05eb4212-635e-4acb-9d4d-375d2804952c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:05.485912 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05eb4212-635e-4acb-9d4d-375d2804952c None admin] RBAC: Adding query filter params (group.id=808ddc3a7fe547139412ee50b7269ea7, role.id=9c769b5c82884766b2c510b1f0f1364c, scope.project.id=69bea7b802b04b689af0d31b020f8554) {{(pid=81449) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:11:05.486222 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05eb4212-635e-4acb-9d4d-375d2804952c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:11:05.487666 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05eb4212-635e-4acb-9d4d-375d2804952c None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:11:05.494504 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 226/455] 10.222.0.15 () {56 vars in 1376 bytes} [Thu Sep 15 12:11:05 2022] GET /identity/v3/role_assignments?group.id=808ddc3a7fe547139412ee50b7269ea7&scope.project.id=69bea7b802b04b689af0d31b020f8554&role.id=9c769b5c82884766b2c510b1f0f1364c => generated 611 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 12:11:08.342411 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79dd8b1e-5ea8-4705-b0c4-3c6bd262361e None None] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:08.342906 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79dd8b1e-5ea8-4705-b0c4-3c6bd262361e None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:08.342906 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79dd8b1e-5ea8-4705-b0c4-3c6bd262361e None None] PATH_INFO: `/` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:08.344692 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 230/456] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 12:11:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:11:08.350812 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182f7b3f-342e-4260-983b-da9d8ce3cf41 None None] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:08.351803 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182f7b3f-342e-4260-983b-da9d8ce3cf41 None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:08.352500 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182f7b3f-342e-4260-983b-da9d8ce3cf41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:08.377791 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.auth.core [None req-182f7b3f-342e-4260-983b-da9d8ce3cf41 None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81449) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:11:08.379370 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-182f7b3f-342e-4260-983b-da9d8ce3cf41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:11:08.481032 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70e2a4b-cb31-4597-a148-5b0b86d65670 None None] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:08.481654 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70e2a4b-cb31-4597-a148-5b0b86d65670 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:08.481834 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 227/457] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 12:11:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 15 12:11:08.482315 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70e2a4b-cb31-4597-a148-5b0b86d65670 None None] PATH_INFO: `/` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:08.483739 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 231/458] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 12:11:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:11:08.489053 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73a5d55-0ec4-4f44-b2dc-efe6893fb8b1 None None] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:08.489345 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73a5d55-0ec4-4f44-b2dc-efe6893fb8b1 None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:08.489623 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73a5d55-0ec4-4f44-b2dc-efe6893fb8b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:08.498455 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:11:08.510343 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.auth.core [None req-f73a5d55-0ec4-4f44-b2dc-efe6893fb8b1 None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81449) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:11:08.511527 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-f73a5d55-0ec4-4f44-b2dc-efe6893fb8b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:11:08.597095 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 228/459] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 12:11:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 12:11:08.674254 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21838483-94fe-48d3-b4fb-dec5ab18af38 None None] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:08.674894 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21838483-94fe-48d3-b4fb-dec5ab18af38 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:08.675362 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21838483-94fe-48d3-b4fb-dec5ab18af38 None None] PATH_INFO: `/` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:08.677171 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 232/460] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 12:11:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:11:08.693783 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c83222-36e5-4164-b464-a7671c4b868a None None] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:08.694073 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c83222-36e5-4164-b464-a7671c4b868a None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:08.694347 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c83222-36e5-4164-b464-a7671c4b868a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:08.718001 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.auth.core [None req-11c83222-36e5-4164-b464-a7671c4b868a None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81449) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:11:08.719322 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-11c83222-36e5-4164-b464-a7671c4b868a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:11:08.793330 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 229/461] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 12:11:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 12:11:08.851224 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21838483-94fe-48d3-b4fb-dec5ab18af38 None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:08.860574 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b8d2a6-be7e-43d4-98ea-b4a99a7b8983 None None] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:08.860885 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b8d2a6-be7e-43d4-98ea-b4a99a7b8983 None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:08.861160 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b8d2a6-be7e-43d4-98ea-b4a99a7b8983 None None] PATH_INFO: `/` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:08.862773 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 230/462] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 12:11:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 15 12:11:08.873570 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14440c5-47c6-4c2b-b781-edebe0061d5c None None] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:08.874097 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14440c5-47c6-4c2b-b781-edebe0061d5c None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:08.874567 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14440c5-47c6-4c2b-b781-edebe0061d5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:08.899972 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.auth.core [None req-c14440c5-47c6-4c2b-b781-edebe0061d5c None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81449) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:11:08.901628 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-c14440c5-47c6-4c2b-b781-edebe0061d5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:11:09.003114 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:11:09.010904 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21838483-94fe-48d3-b4fb-dec5ab18af38 None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:09.021036 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 231/463] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 12:11:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 12:11:09.023795 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14440c5-47c6-4c2b-b781-edebe0061d5c None None] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:09.077956 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320858dc-141b-4c03-90ab-bfa00f58729f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:09.079145 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320858dc-141b-4c03-90ab-bfa00f58729f None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:09.080369 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320858dc-141b-4c03-90ab-bfa00f58729f None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:09.081395 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320858dc-141b-4c03-90ab-bfa00f58729f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:09.087341 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:11:09.105676 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-320858dc-141b-4c03-90ab-bfa00f58729f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:11:09.107077 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-320858dc-141b-4c03-90ab-bfa00f58729f None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:11:09.122313 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application [None req-320858dc-141b-4c03-90ab-bfa00f58729f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 12:11:09.122313 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:11:09.122313 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:11:09.122313 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:11:09.122313 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:11:09.122313 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:11:09.122313 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:11:09.122313 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:11:09.122313 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:11:09.122313 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:11:09.122313 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:11:09.122313 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:11:09.122313 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 12:11:09.122313 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 12:11:09.122313 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 12:11:09.122313 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 12:11:09.123266 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:11:09.123266 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:11:09.123266 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:11:09.123266 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:11:09.123266 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:11:09.123266 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:11:09.123266 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:11:09.123266 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application with Lock( Sep 15 12:11:09.123266 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:11:09.123266 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:11:09.123266 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:11:09.123266 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:11:09.123266 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:11:09.123266 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:11:09.123266 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:11:09.123266 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:11:09.123266 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 12:11:09.124125 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 12:11:09.124125 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 12:11:09.124125 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 15 12:11:09.128560 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 233/464] 10.222.0.15 () {56 vars in 1108 bytes} [Thu Sep 15 12:11:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 278 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:11:09.129439 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320858dc-141b-4c03-90ab-bfa00f58729f None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:09.179981 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14440c5-47c6-4c2b-b781-edebe0061d5c None None] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:09.226908 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc42497-d748-4ea7-9b3a-7058ab85824c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:09.228506 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc42497-d748-4ea7-9b3a-7058ab85824c None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:09.229159 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc42497-d748-4ea7-9b3a-7058ab85824c None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:09.230103 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc42497-d748-4ea7-9b3a-7058ab85824c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:09.234488 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320858dc-141b-4c03-90ab-bfa00f58729f None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:09.244557 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc42497-d748-4ea7-9b3a-7058ab85824c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:11:09.247289 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc42497-d748-4ea7-9b3a-7058ab85824c None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:11:09.268557 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application [None req-afc42497-d748-4ea7-9b3a-7058ab85824c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 12:11:09.268557 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:11:09.268557 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:11:09.268557 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:11:09.268557 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:11:09.268557 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:11:09.268557 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:11:09.268557 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:11:09.268557 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:11:09.268557 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:11:09.268557 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:11:09.268557 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:11:09.268557 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 12:11:09.268557 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 12:11:09.268557 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 12:11:09.268557 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 12:11:09.269525 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:11:09.269525 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:11:09.269525 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:11:09.269525 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:11:09.269525 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:11:09.269525 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:11:09.269525 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:11:09.269525 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application with Lock( Sep 15 12:11:09.269525 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:11:09.269525 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:11:09.269525 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:11:09.269525 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:11:09.269525 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:11:09.269525 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:11:09.269525 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:11:09.269525 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:11:09.269525 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 12:11:09.270382 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 12:11:09.270382 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 12:11:09.270382 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application Sep 15 12:11:09.270974 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a38c90-bd07-4cca-8a99-44ffcd98d8cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:09.271329 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a38c90-bd07-4cca-8a99-44ffcd98d8cb None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:09.271610 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a38c90-bd07-4cca-8a99-44ffcd98d8cb None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:09.271882 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a38c90-bd07-4cca-8a99-44ffcd98d8cb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:09.273331 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 232/465] 10.222.0.15 () {56 vars in 1108 bytes} [Thu Sep 15 12:11:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:11:09.275102 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc42497-d748-4ea7-9b3a-7058ab85824c None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:09.283115 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a38c90-bd07-4cca-8a99-44ffcd98d8cb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:11:09.284570 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a38c90-bd07-4cca-8a99-44ffcd98d8cb None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:11:09.301688 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application [None req-86a38c90-bd07-4cca-8a99-44ffcd98d8cb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 12:11:09.301688 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:11:09.301688 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:11:09.301688 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:11:09.301688 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:11:09.301688 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:11:09.301688 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:11:09.301688 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:11:09.301688 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:11:09.301688 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:11:09.301688 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:11:09.301688 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:11:09.301688 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 12:11:09.301688 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 12:11:09.301688 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 12:11:09.301688 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 12:11:09.303179 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:11:09.303179 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:11:09.303179 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:11:09.303179 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:11:09.303179 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:11:09.303179 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:11:09.303179 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:11:09.303179 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application with Lock( Sep 15 12:11:09.303179 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:11:09.303179 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:11:09.303179 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:11:09.303179 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:11:09.303179 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:11:09.303179 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:11:09.303179 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:11:09.303179 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:11:09.303179 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 12:11:09.304504 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 12:11:09.304504 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 12:11:09.304504 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 15 12:11:09.304504 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 234/466] 10.222.0.15 () {56 vars in 1108 bytes} [Thu Sep 15 12:11:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:11:09.304504 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a38c90-bd07-4cca-8a99-44ffcd98d8cb None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:09.340292 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:11:09.357148 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:11:09.399254 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc42497-d748-4ea7-9b3a-7058ab85824c None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:09.422881 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a38c90-bd07-4cca-8a99-44ffcd98d8cb None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:09.432491 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7060892c-c5fc-43c0-b09c-545c1f6e6c86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:09.433130 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7060892c-c5fc-43c0-b09c-545c1f6e6c86 None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:09.433415 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7060892c-c5fc-43c0-b09c-545c1f6e6c86 None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:09.433692 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7060892c-c5fc-43c0-b09c-545c1f6e6c86 None admin] PATH_INFO: `/v3/domains` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:09.434328 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7060892c-c5fc-43c0-b09c-545c1f6e6c86 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81449) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:11:09.434628 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7060892c-c5fc-43c0-b09c-545c1f6e6c86 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:11:09.435996 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7060892c-c5fc-43c0-b09c-545c1f6e6c86 None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:11:09.453178 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 233/467] 10.222.0.15 () {56 vars in 1109 bytes} [Thu Sep 15 12:11:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:11:09.454019 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7060892c-c5fc-43c0-b09c-545c1f6e6c86 None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:09.465351 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37400626-916a-49b2-8b5f-087e1056a93b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:09.466218 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37400626-916a-49b2-8b5f-087e1056a93b None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:09.466743 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37400626-916a-49b2-8b5f-087e1056a93b None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:09.467222 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37400626-916a-49b2-8b5f-087e1056a93b None admin] PATH_INFO: `/v3/domains` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:09.468160 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37400626-916a-49b2-8b5f-087e1056a93b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:11:09.468703 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37400626-916a-49b2-8b5f-087e1056a93b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:11:09.470214 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37400626-916a-49b2-8b5f-087e1056a93b None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:11:09.485047 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 235/468] 10.222.0.15 () {56 vars in 1109 bytes} [Thu Sep 15 12:11:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:11:09.485687 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37400626-916a-49b2-8b5f-087e1056a93b None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:09.559819 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7060892c-c5fc-43c0-b09c-545c1f6e6c86 None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:09.567053 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:11:09.591598 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:11:09.595302 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37400626-916a-49b2-8b5f-087e1056a93b None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:09.598298 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb94c93-62cd-4c91-8ce8-147f6b32faba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:09.598916 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb94c93-62cd-4c91-8ce8-147f6b32faba None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:09.599199 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb94c93-62cd-4c91-8ce8-147f6b32faba None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:09.599496 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb94c93-62cd-4c91-8ce8-147f6b32faba None admin] PATH_INFO: `/v3/domains` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:09.600137 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb94c93-62cd-4c91-8ce8-147f6b32faba None admin] RBAC: Adding query filter params (name=Default) {{(pid=81449) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:11:09.600439 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb94c93-62cd-4c91-8ce8-147f6b32faba None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:11:09.601856 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbb94c93-62cd-4c91-8ce8-147f6b32faba None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:11:09.614292 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 234/469] 10.222.0.15 () {56 vars in 1109 bytes} [Thu Sep 15 12:11:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:11:09.615107 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb94c93-62cd-4c91-8ce8-147f6b32faba None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:09.641045 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1f9075-828a-4d42-85ed-8180a7cbce65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:09.641702 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1f9075-828a-4d42-85ed-8180a7cbce65 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:09.641981 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1f9075-828a-4d42-85ed-8180a7cbce65 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:09.642252 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1f9075-828a-4d42-85ed-8180a7cbce65 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:09.655039 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1f9075-828a-4d42-85ed-8180a7cbce65 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:11:09.656517 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1f9075-828a-4d42-85ed-8180a7cbce65 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:11:09.672236 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application [None req-1b1f9075-828a-4d42-85ed-8180a7cbce65 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 12:11:09.672236 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:11:09.672236 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:11:09.672236 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:11:09.672236 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:11:09.672236 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:11:09.672236 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:11:09.672236 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:11:09.672236 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:11:09.672236 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:11:09.672236 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:11:09.672236 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:11:09.672236 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 12:11:09.672236 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 12:11:09.672236 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 12:11:09.672236 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 12:11:09.673193 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:11:09.673193 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:11:09.673193 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:11:09.673193 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:11:09.673193 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:11:09.673193 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:11:09.673193 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:11:09.673193 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application with Lock( Sep 15 12:11:09.673193 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:11:09.673193 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:11:09.673193 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:11:09.673193 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:11:09.673193 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:11:09.673193 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:11:09.673193 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:11:09.673193 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:11:09.673193 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 12:11:09.674046 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 12:11:09.674046 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 12:11:09.674046 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 15 12:11:09.674046 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 236/470] 10.222.0.15 () {56 vars in 1108 bytes} [Thu Sep 15 12:11:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:11:09.674257 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1f9075-828a-4d42-85ed-8180a7cbce65 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:09.725070 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb94c93-62cd-4c91-8ce8-147f6b32faba None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:09.758705 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea278eb-48cd-4463-b0ce-a1ff7b791cf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:09.759311 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea278eb-48cd-4463-b0ce-a1ff7b791cf4 None admin] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:09.759588 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea278eb-48cd-4463-b0ce-a1ff7b791cf4 None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:09.759857 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea278eb-48cd-4463-b0ce-a1ff7b791cf4 None admin] PATH_INFO: `/v3/users` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:09.760494 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea278eb-48cd-4463-b0ce-a1ff7b791cf4 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:11:09.761999 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ea278eb-48cd-4463-b0ce-a1ff7b791cf4 None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:11:09.778388 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1f9075-828a-4d42-85ed-8180a7cbce65 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:09.782861 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 235/471] 10.222.0.15 () {60 vars in 1132 bytes} [Thu Sep 15 12:11:09 2022] POST /identity/v3/users => generated 246 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 12:11:09.783773 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea278eb-48cd-4463-b0ce-a1ff7b791cf4 None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:09.811216 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:11:09.826788 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3069a3-cd7c-4f32-ae89-a5abc0996858 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:09.827411 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3069a3-cd7c-4f32-ae89-a5abc0996858 None admin] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:09.827697 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3069a3-cd7c-4f32-ae89-a5abc0996858 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:09.827971 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3069a3-cd7c-4f32-ae89-a5abc0996858 None admin] PATH_INFO: `/v3/users` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:09.828636 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3069a3-cd7c-4f32-ae89-a5abc0996858 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:11:09.830120 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3069a3-cd7c-4f32-ae89-a5abc0996858 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:11:09.854011 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 237/472] 10.222.0.15 () {60 vars in 1132 bytes} [Thu Sep 15 12:11:09 2022] POST /identity/v3/users => generated 248 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 12:11:09.855497 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3069a3-cd7c-4f32-ae89-a5abc0996858 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:09.868791 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:11:09.932033 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ea278eb-48cd-4463-b0ce-a1ff7b791cf4 None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:09.986603 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5bbf51-a51b-4be5-9a67-4f58de9a82dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:09.987245 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5bbf51-a51b-4be5-9a67-4f58de9a82dd None admin] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:09.987530 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5bbf51-a51b-4be5-9a67-4f58de9a82dd None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:09.987807 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5bbf51-a51b-4be5-9a67-4f58de9a82dd None admin] PATH_INFO: `/v3/users` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:09.988639 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5bbf51-a51b-4be5-9a67-4f58de9a82dd None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:11:09.991270 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5bbf51-a51b-4be5-9a67-4f58de9a82dd None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:11:09.992309 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3069a3-cd7c-4f32-ae89-a5abc0996858 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:10.021360 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 236/473] 10.222.0.15 () {60 vars in 1132 bytes} [Thu Sep 15 12:11:09 2022] POST /identity/v3/users => generated 248 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 12:11:10.022782 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8bbfd9-4217-4a61-98f4-2be240bd5c07 None None] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:10.023068 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8bbfd9-4217-4a61-98f4-2be240bd5c07 None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:10.023341 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8bbfd9-4217-4a61-98f4-2be240bd5c07 None None] PATH_INFO: `/` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:10.028901 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 237/474] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 12:11:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:11:10.030933 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd98cdd6-dbfe-4f1c-b5c4-e86a6443df21 None None] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:10.031226 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd98cdd6-dbfe-4f1c-b5c4-e86a6443df21 None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:10.031499 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd98cdd6-dbfe-4f1c-b5c4-e86a6443df21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:10.045453 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a864aca-f771-4800-b75a-630bc20e9ee4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:10.047575 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a864aca-f771-4800-b75a-630bc20e9ee4 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:10.047871 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a864aca-f771-4800-b75a-630bc20e9ee4 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:10.048163 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a864aca-f771-4800-b75a-630bc20e9ee4 None admin] PATH_INFO: `/v3/domains` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:10.049873 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a864aca-f771-4800-b75a-630bc20e9ee4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:11:10.049873 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a864aca-f771-4800-b75a-630bc20e9ee4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:11:10.050946 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a864aca-f771-4800-b75a-630bc20e9ee4 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:11:10.057719 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.auth.core [None req-cd98cdd6-dbfe-4f1c-b5c4-e86a6443df21 None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81449) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:11:10.059342 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-cd98cdd6-dbfe-4f1c-b5c4-e86a6443df21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:11:10.071261 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 238/475] 10.222.0.15 () {56 vars in 1109 bytes} [Thu Sep 15 12:11:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:11:10.077481 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a864aca-f771-4800-b75a-630bc20e9ee4 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:10.159562 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 238/476] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 12:11:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 15 12:11:10.170481 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:11:10.214313 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a864aca-f771-4800-b75a-630bc20e9ee4 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:10.256627 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed3cea0-2848-4a0f-b443-4f97c0238b4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:10.257737 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed3cea0-2848-4a0f-b443-4f97c0238b4e None admin] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:10.258217 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed3cea0-2848-4a0f-b443-4f97c0238b4e None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:10.258686 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed3cea0-2848-4a0f-b443-4f97c0238b4e None admin] PATH_INFO: `/v3/users` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:10.259535 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed3cea0-2848-4a0f-b443-4f97c0238b4e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:11:10.261200 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed3cea0-2848-4a0f-b443-4f97c0238b4e None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:11:10.293492 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 239/477] 10.222.0.15 () {60 vars in 1132 bytes} [Thu Sep 15 12:11:10 2022] POST /identity/v3/users => generated 249 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 15 12:11:10.471110 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd98cdd6-dbfe-4f1c-b5c4-e86a6443df21 None None] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:10.482385 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:11:10.580914 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd98cdd6-dbfe-4f1c-b5c4-e86a6443df21 None None] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:10.620604 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85ea244-4c32-43b2-ac5d-7b20422dedf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:10.621252 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85ea244-4c32-43b2-ac5d-7b20422dedf5 None admin] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:10.621530 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85ea244-4c32-43b2-ac5d-7b20422dedf5 None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:10.621798 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f85ea244-4c32-43b2-ac5d-7b20422dedf5 None admin] PATH_INFO: `/v3/roles` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:10.622466 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f85ea244-4c32-43b2-ac5d-7b20422dedf5 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:11:10.623889 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f85ea244-4c32-43b2-ac5d-7b20422dedf5 None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:11:10.633076 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.sql.core [None req-f85ea244-4c32-43b2-ac5d-7b20422dedf5 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 15 12:11:10.633076 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Sep 15 12:11:10.633076 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [parameters: {'id': 'f7be5544e7d845618d27431ddf7f3b3d', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Sep 15 12:11:10.633076 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81449) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 15 12:11:10.650309 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application [None req-f85ea244-4c32-43b2-ac5d-7b20422dedf5 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Sep 15 12:11:10.650309 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:11:10.650309 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 15 12:11:10.650309 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 15 12:11:10.650309 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 15 12:11:10.650309 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 15 12:11:10.650309 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 15 12:11:10.650309 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application result = self._query(query) Sep 15 12:11:10.650309 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 15 12:11:10.650309 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application conn.query(q) Sep 15 12:11:10.650309 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 15 12:11:10.650309 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 15 12:11:10.650309 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 15 12:11:10.650309 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application result.read() Sep 15 12:11:10.650309 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 15 12:11:10.651204 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 15 12:11:10.651204 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 15 12:11:10.651204 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 15 12:11:10.651204 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 15 12:11:10.651204 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 15 12:11:10.651204 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 15 12:11:10.651204 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 15 12:11:10.651204 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 15 12:11:10.651204 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application Sep 15 12:11:10.651204 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 15 12:11:10.651204 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application Sep 15 12:11:10.651204 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:11:10.651204 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 15 12:11:10.651204 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 15 12:11:10.651204 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Sep 15 12:11:10.651204 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return ref.to_dict() Sep 15 12:11:10.651204 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 15 12:11:10.651204 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application next(self.gen) Sep 15 12:11:10.652556 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 15 12:11:10.652556 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application yield resource Sep 15 12:11:10.652556 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 15 12:11:10.652556 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application next(self.gen) Sep 15 12:11:10.652556 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 15 12:11:10.652556 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application self.session.rollback() Sep 15 12:11:10.652556 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 15 12:11:10.652556 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application self.force_reraise() Sep 15 12:11:10.652556 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 15 12:11:10.652556 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application raise self.value Sep 15 12:11:10.652556 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 15 12:11:10.652556 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 15 12:11:10.652556 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 15 12:11:10.652556 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application session.commit() Sep 15 12:11:10.652556 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 15 12:11:10.652556 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 15 12:11:10.652556 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 15 12:11:10.653719 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application self._prepare_impl() Sep 15 12:11:10.653719 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 15 12:11:10.653719 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application self.session.flush() Sep 15 12:11:10.653719 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 15 12:11:10.653719 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application self._flush(objects) Sep 15 12:11:10.653719 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 15 12:11:10.653719 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 15 12:11:10.653719 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 15 12:11:10.653719 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application compat.raise_( Sep 15 12:11:10.653719 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 15 12:11:10.653719 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application raise exception Sep 15 12:11:10.653719 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 15 12:11:10.653719 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application flush_context.execute() Sep 15 12:11:10.653719 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 15 12:11:10.653719 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application rec.execute(self) Sep 15 12:11:10.653719 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 15 12:11:10.653719 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 15 12:11:10.654592 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 15 12:11:10.654592 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 15 12:11:10.654592 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Sep 15 12:11:10.654592 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application c = connection._execute_20( Sep 15 12:11:10.654592 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 15 12:11:10.654592 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 15 12:11:10.654592 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 15 12:11:10.654592 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 15 12:11:10.654592 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 15 12:11:10.654592 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 15 12:11:10.654592 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 15 12:11:10.654592 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 15 12:11:10.654592 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 15 12:11:10.654592 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 15 12:11:10.654592 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 15 12:11:10.654592 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application raise exception Sep 15 12:11:10.655392 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 15 12:11:10.655392 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 15 12:11:10.655392 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 15 12:11:10.655392 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 15 12:11:10.655392 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 15 12:11:10.655392 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application result = self._query(query) Sep 15 12:11:10.655392 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 15 12:11:10.655392 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application conn.query(q) Sep 15 12:11:10.655392 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 15 12:11:10.655392 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 15 12:11:10.655392 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 15 12:11:10.655392 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application result.read() Sep 15 12:11:10.655392 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 15 12:11:10.655392 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 15 12:11:10.655392 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 15 12:11:10.655392 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 15 12:11:10.656181 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 15 12:11:10.656181 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 15 12:11:10.656181 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 15 12:11:10.656181 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 15 12:11:10.656181 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Sep 15 12:11:10.656181 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Sep 15 12:11:10.656181 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application [parameters: {'id': 'f7be5544e7d845618d27431ddf7f3b3d', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Sep 15 12:11:10.656181 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 15 12:11:10.656181 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application Sep 15 12:11:10.656181 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 15 12:11:10.656181 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application Sep 15 12:11:10.656181 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:11:10.656181 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:11:10.656181 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:11:10.656181 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:11:10.656181 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:11:10.660626 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:11:10.660626 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:11:10.660626 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:11:10.660626 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:11:10.660626 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:11:10.660626 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:11:10.660626 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Sep 15 12:11:10.660626 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Sep 15 12:11:10.660626 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:11:10.660626 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:11:10.660626 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Sep 15 12:11:10.660626 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Sep 15 12:11:10.660626 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 15 12:11:10.660626 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 15 12:11:10.660626 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Sep 15 12:11:10.660626 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application Sep 15 12:11:10.662986 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 239/478] 10.222.0.15 () {60 vars in 1131 bytes} [Thu Sep 15 12:11:10 2022] POST /identity/v3/roles => generated 144 bytes in 181 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:11:10.663108 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed3cea0-2848-4a0f-b443-4f97c0238b4e None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:10.793095 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed3cea0-2848-4a0f-b443-4f97c0238b4e None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:10.825252 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:11:10.836194 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62588a5f-06bc-4664-8c39-24603744b1c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:10.836877 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62588a5f-06bc-4664-8c39-24603744b1c3 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:10.837507 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62588a5f-06bc-4664-8c39-24603744b1c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:10.837785 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62588a5f-06bc-4664-8c39-24603744b1c3 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:10.848892 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62588a5f-06bc-4664-8c39-24603744b1c3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 12:11:10.854203 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application [None req-62588a5f-06bc-4664-8c39-24603744b1c3 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 15 12:11:10.854203 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:11:10.854203 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:11:10.854203 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:11:10.854203 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:11:10.854203 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:11:10.854203 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:11:10.854203 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:11:10.854203 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:11:10.854203 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:11:10.854203 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:11:10.854203 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:11:10.854203 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 12:11:10.854203 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 12:11:10.854203 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 12:11:10.854203 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 12:11:10.855376 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 12:11:10.855376 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 12:11:10.855376 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 12:11:10.855376 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 12:11:10.855376 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:11:10.855376 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:11:10.855376 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:11:10.855376 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:11:10.855376 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:11:10.855376 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:11:10.855376 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:11:10.855376 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application with Lock( Sep 15 12:11:10.855376 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:11:10.855376 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:11:10.855376 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:11:10.855376 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:11:10.856594 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:11:10.856594 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:11:10.856594 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:11:10.856594 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:11:10.856594 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 12:11:10.856594 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 12:11:10.856594 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 12:11:10.856594 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 12:11:10.856594 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 12:11:10.856594 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 12:11:10.856594 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Sep 15 12:11:10.856594 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 15 12:11:10.856594 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 240/479] 10.222.0.15 () {56 vars in 1114 bytes} [Thu Sep 15 12:11:10 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:11:10.861777 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85ea244-4c32-43b2-ac5d-7b20422dedf5 None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:10.935828 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:11:10.981996 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f85ea244-4c32-43b2-ac5d-7b20422dedf5 None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:11.030333 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4669a45a-8785-4cea-a3bb-9977092107a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:11.030958 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4669a45a-8785-4cea-a3bb-9977092107a8 None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:11.031238 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4669a45a-8785-4cea-a3bb-9977092107a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:11.031514 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4669a45a-8785-4cea-a3bb-9977092107a8 None admin] PATH_INFO: `/v3/roles` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:11.032182 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4669a45a-8785-4cea-a3bb-9977092107a8 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81449) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:11:11.032485 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4669a45a-8785-4cea-a3bb-9977092107a8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:11:11.033883 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4669a45a-8785-4cea-a3bb-9977092107a8 None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:11:11.049290 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 240/480] 10.222.0.15 () {56 vars in 1111 bytes} [Thu Sep 15 12:11:10 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:11:12.510162 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0a7b4a-ac8c-4038-b5b7-50407aa427d5 None None] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:12.510705 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0a7b4a-ac8c-4038-b5b7-50407aa427d5 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:12.510705 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0a7b4a-ac8c-4038-b5b7-50407aa427d5 None None] PATH_INFO: `/` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:12.512037 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 241/481] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 12:11:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:11:12.528522 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac56ded-a524-427b-80c3-0791d45f619b None None] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:12.529409 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac56ded-a524-427b-80c3-0791d45f619b None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:12.529917 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac56ded-a524-427b-80c3-0791d45f619b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:12.557972 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.auth.core [None req-dac56ded-a524-427b-80c3-0791d45f619b None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81449) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:11:12.559559 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-dac56ded-a524-427b-80c3-0791d45f619b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:11:12.592533 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27655c70-398b-4675-afef-f8dea705d1dc None None] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:12.592533 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27655c70-398b-4675-afef-f8dea705d1dc None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:12.592533 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27655c70-398b-4675-afef-f8dea705d1dc None None] PATH_INFO: `/` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:12.592533 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 242/482] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 12:11:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:11:12.598457 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580cc889-3f38-4c60-a51c-b68aed2e5fda None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:12.598741 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580cc889-3f38-4c60-a51c-b68aed2e5fda None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:12.599018 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580cc889-3f38-4c60-a51c-b68aed2e5fda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:12.624571 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-580cc889-3f38-4c60-a51c-b68aed2e5fda None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:11:12.626527 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-580cc889-3f38-4c60-a51c-b68aed2e5fda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:11:12.657988 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 241/483] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 12:11:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 12:11:12.718958 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 243/484] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 12:11:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 15 12:11:12.866624 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa604a0a-da8f-405a-aeec-9ec84b537892 None None] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:12.866812 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa604a0a-da8f-405a-aeec-9ec84b537892 None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:12.867086 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa604a0a-da8f-405a-aeec-9ec84b537892 None None] PATH_INFO: `/` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:12.868674 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 242/485] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 12:11:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:11:12.873845 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bdccf6-b2db-4f83-8f3b-01064844cd84 None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:12.874137 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bdccf6-b2db-4f83-8f3b-01064844cd84 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:12.874417 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bdccf6-b2db-4f83-8f3b-01064844cd84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:12.901102 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-c4bdccf6-b2db-4f83-8f3b-01064844cd84 None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:11:12.902293 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-c4bdccf6-b2db-4f83-8f3b-01064844cd84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:11:13.011051 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 244/486] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 12:11:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 12:11:13.068877 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa604a0a-da8f-405a-aeec-9ec84b537892 None None] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:13.106744 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4bdccf6-b2db-4f83-8f3b-01064844cd84 None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:13.185797 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:11:13.190655 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:11:13.243612 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa604a0a-da8f-405a-aeec-9ec84b537892 None None] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:13.244565 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4bdccf6-b2db-4f83-8f3b-01064844cd84 None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:13.282689 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b5088c-91e3-4aee-b16f-a45550f9f637 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:13.283625 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b5088c-91e3-4aee-b16f-a45550f9f637 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:13.284083 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b5088c-91e3-4aee-b16f-a45550f9f637 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:13.284541 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b5088c-91e3-4aee-b16f-a45550f9f637 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:13.288025 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ccf379-7c5a-4059-a156-f79e21ef5f44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:13.288655 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ccf379-7c5a-4059-a156-f79e21ef5f44 None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:13.288984 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ccf379-7c5a-4059-a156-f79e21ef5f44 None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:13.289259 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ccf379-7c5a-4059-a156-f79e21ef5f44 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:13.296176 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b5088c-91e3-4aee-b16f-a45550f9f637 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 12:11:13.301179 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ccf379-7c5a-4059-a156-f79e21ef5f44 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 12:11:13.301594 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application [None req-85b5088c-91e3-4aee-b16f-a45550f9f637 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 15 12:11:13.301594 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:11:13.301594 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:11:13.301594 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:11:13.301594 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:11:13.301594 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:11:13.301594 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:11:13.301594 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:11:13.301594 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:11:13.301594 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:11:13.301594 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:11:13.301594 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:11:13.301594 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 12:11:13.301594 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 12:11:13.301594 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 12:11:13.301594 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 12:11:13.302534 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 12:11:13.302534 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 12:11:13.302534 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 12:11:13.302534 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 12:11:13.302534 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:11:13.302534 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:11:13.302534 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:11:13.302534 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:11:13.302534 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:11:13.302534 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:11:13.302534 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:11:13.302534 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application with Lock( Sep 15 12:11:13.302534 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:11:13.302534 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:11:13.302534 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:11:13.302534 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:11:13.303348 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:11:13.303348 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:11:13.303348 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:11:13.303348 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:11:13.303348 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 12:11:13.303348 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 12:11:13.303348 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 12:11:13.303348 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 12:11:13.303348 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 12:11:13.303348 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 12:11:13.303348 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 15 12:11:13.303348 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 15 12:11:13.305615 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 245/487] 10.222.0.15 () {56 vars in 1102 bytes} [Thu Sep 15 12:11:13 2022] GET /identity/v3/roles/service => generated 85 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 15 12:11:13.306016 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application [None req-e7ccf379-7c5a-4059-a156-f79e21ef5f44 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 15 12:11:13.306016 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:11:13.306016 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:11:13.306016 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:11:13.306016 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:11:13.306016 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:11:13.306016 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:11:13.306016 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:11:13.306016 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:11:13.306016 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:11:13.306016 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:11:13.306016 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:11:13.306016 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 12:11:13.306016 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 12:11:13.306016 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 12:11:13.306016 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 12:11:13.306948 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 12:11:13.306948 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 12:11:13.306948 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 12:11:13.306948 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 12:11:13.306948 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:11:13.306948 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:11:13.306948 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:11:13.306948 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:11:13.306948 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:11:13.306948 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:11:13.306948 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:11:13.306948 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application with Lock( Sep 15 12:11:13.306948 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:11:13.306948 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:11:13.306948 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:11:13.306948 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:11:13.307746 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:11:13.307746 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:11:13.307746 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:11:13.307746 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:11:13.307746 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 12:11:13.307746 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 12:11:13.307746 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 12:11:13.307746 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 12:11:13.307746 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 12:11:13.307746 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 12:11:13.307746 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 15 12:11:13.307746 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application Sep 15 12:11:13.307746 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 243/488] 10.222.0.15 () {56 vars in 1102 bytes} [Thu Sep 15 12:11:13 2022] GET /identity/v3/roles/service => generated 85 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:11:13.309934 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410553e3-5628-455a-b6d8-0e35db672712 None None] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:13.310397 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410553e3-5628-455a-b6d8-0e35db672712 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:13.310842 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410553e3-5628-455a-b6d8-0e35db672712 None None] PATH_INFO: `/` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:13.310942 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ccf379-7c5a-4059-a156-f79e21ef5f44 None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:13.312654 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 246/489] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 12:11:13 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:11:13.315053 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410553e3-5628-455a-b6d8-0e35db672712 None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:13.427042 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ccf379-7c5a-4059-a156-f79e21ef5f44 None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:13.432328 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410553e3-5628-455a-b6d8-0e35db672712 None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:13.434692 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:11:13.455069 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:11:13.468540 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c7a1f8-66b9-48ef-b31c-c1b920d2d364 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:13.469098 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f90a09-0e2d-4dab-ad4e-70ca0301dd84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:13.469219 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c7a1f8-66b9-48ef-b31c-c1b920d2d364 None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:13.469457 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c7a1f8-66b9-48ef-b31c-c1b920d2d364 None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:13.469709 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f90a09-0e2d-4dab-ad4e-70ca0301dd84 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:13.469819 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c7a1f8-66b9-48ef-b31c-c1b920d2d364 None admin] PATH_INFO: `/v3/roles` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:13.470053 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f90a09-0e2d-4dab-ad4e-70ca0301dd84 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:13.470337 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f90a09-0e2d-4dab-ad4e-70ca0301dd84 None admin] PATH_INFO: `/v3/roles` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:13.470430 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c7a1f8-66b9-48ef-b31c-c1b920d2d364 None admin] RBAC: Adding query filter params (name=service) {{(pid=81449) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:11:13.470668 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c7a1f8-66b9-48ef-b31c-c1b920d2d364 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:11:13.470995 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f90a09-0e2d-4dab-ad4e-70ca0301dd84 None admin] RBAC: Adding query filter params (name=service) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:11:13.471301 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f90a09-0e2d-4dab-ad4e-70ca0301dd84 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:11:13.472057 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c7a1f8-66b9-48ef-b31c-c1b920d2d364 None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:11:13.472648 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f90a09-0e2d-4dab-ad4e-70ca0301dd84 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:11:13.486517 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 247/490] 10.222.0.15 () {56 vars in 1103 bytes} [Thu Sep 15 12:11:13 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 12:11:13.486517 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24b4970-ddf1-41a9-852c-0c569f6ffb21 None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:13.486517 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24b4970-ddf1-41a9-852c-0c569f6ffb21 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:13.486517 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24b4970-ddf1-41a9-852c-0c569f6ffb21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:13.486874 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 244/491] 10.222.0.15 () {56 vars in 1103 bytes} [Thu Sep 15 12:11:13 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 12:11:13.487394 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c7a1f8-66b9-48ef-b31c-c1b920d2d364 None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:13.501345 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-d24b4970-ddf1-41a9-852c-0c569f6ffb21 None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:11:13.502536 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-d24b4970-ddf1-41a9-852c-0c569f6ffb21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:11:13.576900 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 248/492] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 12:11:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 12:11:13.578539 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24b4970-ddf1-41a9-852c-0c569f6ffb21 None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:13.611562 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c7a1f8-66b9-48ef-b31c-c1b920d2d364 None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:13.648902 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b7b265-1e8a-4349-8063-dee090296716 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:13.649541 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b7b265-1e8a-4349-8063-dee090296716 None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:13.650124 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b7b265-1e8a-4349-8063-dee090296716 None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:13.650233 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b7b265-1e8a-4349-8063-dee090296716 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:13.661074 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b7b265-1e8a-4349-8063-dee090296716 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 12:11:13.666761 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application [None req-01b7b265-1e8a-4349-8063-dee090296716 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 15 12:11:13.666761 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:11:13.666761 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:11:13.666761 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:11:13.666761 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:11:13.666761 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:11:13.666761 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:11:13.666761 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:11:13.666761 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:11:13.666761 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:11:13.666761 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:11:13.666761 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:11:13.666761 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 12:11:13.666761 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 12:11:13.666761 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 12:11:13.666761 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 12:11:13.667677 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 12:11:13.667677 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 12:11:13.667677 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 12:11:13.667677 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 12:11:13.667677 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:11:13.667677 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:11:13.667677 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:11:13.667677 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:11:13.667677 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:11:13.667677 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:11:13.667677 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:11:13.667677 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application with Lock( Sep 15 12:11:13.667677 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:11:13.667677 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:11:13.667677 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:11:13.667677 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:11:13.668490 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:11:13.668490 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:11:13.668490 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:11:13.668490 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:11:13.668490 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 12:11:13.668490 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 12:11:13.668490 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 12:11:13.668490 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 12:11:13.668490 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 12:11:13.668490 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 12:11:13.668490 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 15 12:11:13.668490 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application Sep 15 12:11:13.672602 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 245/493] 10.222.0.15 () {56 vars in 1102 bytes} [Thu Sep 15 12:11:13 2022] GET /identity/v3/roles/service => generated 85 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:11:13.673470 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b7b265-1e8a-4349-8063-dee090296716 None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:13.688956 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:11:13.696847 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24b4970-ddf1-41a9-852c-0c569f6ffb21 None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:13.726699 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:11:13.738093 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d297e5a-cff5-4a9b-a511-a82fb51656ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:13.739071 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d297e5a-cff5-4a9b-a511-a82fb51656ed None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:13.739457 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d297e5a-cff5-4a9b-a511-a82fb51656ed None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:13.740073 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d297e5a-cff5-4a9b-a511-a82fb51656ed None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:13.751706 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d297e5a-cff5-4a9b-a511-a82fb51656ed None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:11:13.753357 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d297e5a-cff5-4a9b-a511-a82fb51656ed None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:11:13.770668 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application [None req-1d297e5a-cff5-4a9b-a511-a82fb51656ed None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 12:11:13.770668 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:11:13.770668 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:11:13.770668 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:11:13.770668 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:11:13.770668 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:11:13.770668 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:11:13.770668 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:11:13.770668 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:11:13.770668 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:11:13.770668 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:11:13.770668 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:11:13.770668 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 12:11:13.770668 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 12:11:13.770668 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 12:11:13.770668 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 12:11:13.772137 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:11:13.772137 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:11:13.772137 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:11:13.772137 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:11:13.772137 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:11:13.772137 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:11:13.772137 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:11:13.772137 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application with Lock( Sep 15 12:11:13.772137 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:11:13.772137 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:11:13.772137 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:11:13.772137 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:11:13.772137 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:11:13.772137 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:11:13.772137 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:11:13.772137 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:11:13.772137 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 12:11:13.773118 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 12:11:13.773118 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 12:11:13.773118 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 15 12:11:13.777305 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 249/494] 10.222.0.15 () {56 vars in 1108 bytes} [Thu Sep 15 12:11:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:11:13.778156 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d297e5a-cff5-4a9b-a511-a82fb51656ed None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:13.800723 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b7b265-1e8a-4349-8063-dee090296716 None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:13.839822 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7254bb5-7af1-4483-bbed-270a9abfee55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:13.840456 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7254bb5-7af1-4483-bbed-270a9abfee55 None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:13.840735 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7254bb5-7af1-4483-bbed-270a9abfee55 None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:13.841036 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7254bb5-7af1-4483-bbed-270a9abfee55 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:13.852407 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7254bb5-7af1-4483-bbed-270a9abfee55 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:11:13.853903 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7254bb5-7af1-4483-bbed-270a9abfee55 None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:11:13.871258 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application [None req-e7254bb5-7af1-4483-bbed-270a9abfee55 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 12:11:13.871258 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:11:13.871258 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:11:13.871258 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:11:13.871258 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:11:13.871258 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:11:13.871258 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:11:13.871258 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:11:13.871258 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:11:13.871258 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:11:13.871258 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:11:13.871258 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:11:13.871258 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 12:11:13.871258 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 12:11:13.871258 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 12:11:13.871258 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 12:11:13.872194 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:11:13.872194 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:11:13.872194 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:11:13.872194 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:11:13.872194 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:11:13.872194 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:11:13.872194 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:11:13.872194 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application with Lock( Sep 15 12:11:13.872194 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:11:13.872194 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:11:13.872194 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:11:13.872194 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:11:13.872194 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:11:13.872194 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:11:13.872194 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:11:13.872194 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:11:13.872194 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 12:11:13.873089 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 12:11:13.873089 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 12:11:13.873089 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application Sep 15 12:11:13.873089 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 246/495] 10.222.0.15 () {56 vars in 1108 bytes} [Thu Sep 15 12:11:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:11:13.877236 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7254bb5-7af1-4483-bbed-270a9abfee55 None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:13.907285 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d297e5a-cff5-4a9b-a511-a82fb51656ed None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:13.940938 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155f121f-07dd-492a-83d7-a816aea39cd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:13.941554 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155f121f-07dd-492a-83d7-a816aea39cd9 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:13.941832 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155f121f-07dd-492a-83d7-a816aea39cd9 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:13.942103 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155f121f-07dd-492a-83d7-a816aea39cd9 None admin] PATH_INFO: `/v3/roles` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:13.942756 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-155f121f-07dd-492a-83d7-a816aea39cd9 None admin] RBAC: Adding query filter params (name=service) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:11:13.943082 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-155f121f-07dd-492a-83d7-a816aea39cd9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:11:13.944437 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-155f121f-07dd-492a-83d7-a816aea39cd9 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:11:13.952217 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 250/496] 10.222.0.15 () {56 vars in 1103 bytes} [Thu Sep 15 12:11:13 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:11:13.953047 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155f121f-07dd-492a-83d7-a816aea39cd9 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:13.969234 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:11:13.980140 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:11:13.984371 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7254bb5-7af1-4483-bbed-270a9abfee55 None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:14.017451 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2be358-2256-4565-9539-05f20db7a478 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:14.018053 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2be358-2256-4565-9539-05f20db7a478 None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:14.018333 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2be358-2256-4565-9539-05f20db7a478 None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:14.018605 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2be358-2256-4565-9539-05f20db7a478 None admin] PATH_INFO: `/v3/domains` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:14.019238 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a2be358-2256-4565-9539-05f20db7a478 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81449) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:11:14.019538 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a2be358-2256-4565-9539-05f20db7a478 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:11:14.020948 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a2be358-2256-4565-9539-05f20db7a478 None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:11:14.032830 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 247/497] 10.222.0.15 () {56 vars in 1109 bytes} [Thu Sep 15 12:11:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:11:14.033625 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2be358-2256-4565-9539-05f20db7a478 None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:14.057007 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155f121f-07dd-492a-83d7-a816aea39cd9 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:14.089951 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533045ab-31a5-487e-b5d1-a7bef5859c75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:14.090579 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533045ab-31a5-487e-b5d1-a7bef5859c75 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:14.090877 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533045ab-31a5-487e-b5d1-a7bef5859c75 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:14.091128 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533045ab-31a5-487e-b5d1-a7bef5859c75 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:14.102293 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533045ab-31a5-487e-b5d1-a7bef5859c75 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 12:11:14.107869 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application [None req-533045ab-31a5-487e-b5d1-a7bef5859c75 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 15 12:11:14.107869 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:11:14.107869 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:11:14.107869 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:11:14.107869 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:11:14.107869 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:11:14.107869 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:11:14.107869 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:11:14.107869 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:11:14.107869 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:11:14.107869 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:11:14.107869 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:11:14.107869 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 12:11:14.107869 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 12:11:14.107869 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 12:11:14.107869 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 12:11:14.108954 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 12:11:14.108954 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 12:11:14.108954 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 12:11:14.108954 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 12:11:14.108954 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:11:14.108954 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:11:14.108954 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:11:14.108954 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:11:14.108954 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:11:14.108954 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:11:14.108954 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:11:14.108954 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application with Lock( Sep 15 12:11:14.108954 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:11:14.108954 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:11:14.108954 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:11:14.108954 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:11:14.109949 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:11:14.109949 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:11:14.109949 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:11:14.109949 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:11:14.109949 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 12:11:14.109949 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 12:11:14.109949 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 12:11:14.109949 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 12:11:14.109949 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 12:11:14.109949 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 12:11:14.109949 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 15 12:11:14.109949 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 15 12:11:14.112890 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 251/498] 10.222.0.15 () {56 vars in 1102 bytes} [Thu Sep 15 12:11:13 2022] GET /identity/v3/roles/service => generated 85 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:11:14.114251 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65257f9a-2553-4a90-af1d-99213f315762 None None] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:14.114564 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65257f9a-2553-4a90-af1d-99213f315762 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:14.114837 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65257f9a-2553-4a90-af1d-99213f315762 None None] PATH_INFO: `/` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:14.116088 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 252/499] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 12:11:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:11:14.116890 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65257f9a-2553-4a90-af1d-99213f315762 None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:14.135752 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2be358-2256-4565-9539-05f20db7a478 None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:14.168628 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9400ec27-054c-4647-a163-2eef957b18ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:14.169262 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9400ec27-054c-4647-a163-2eef957b18ae None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:14.169547 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9400ec27-054c-4647-a163-2eef957b18ae None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:14.169820 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9400ec27-054c-4647-a163-2eef957b18ae None admin] PATH_INFO: `/v3/domains` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:14.170454 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9400ec27-054c-4647-a163-2eef957b18ae None admin] RBAC: Adding query filter params (name=Default) {{(pid=81449) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:11:14.170759 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9400ec27-054c-4647-a163-2eef957b18ae None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:11:14.172119 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9400ec27-054c-4647-a163-2eef957b18ae None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:11:14.182979 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 248/500] 10.222.0.15 () {56 vars in 1109 bytes} [Thu Sep 15 12:11:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:11:14.183759 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9400ec27-054c-4647-a163-2eef957b18ae None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:14.185682 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:11:14.206456 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:11:14.218339 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65257f9a-2553-4a90-af1d-99213f315762 None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:14.250424 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76b7925-1d11-48fa-82eb-53b5aa304bb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:14.250992 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76b7925-1d11-48fa-82eb-53b5aa304bb3 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:14.251272 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76b7925-1d11-48fa-82eb-53b5aa304bb3 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:14.251542 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76b7925-1d11-48fa-82eb-53b5aa304bb3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:14.264486 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c76b7925-1d11-48fa-82eb-53b5aa304bb3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:11:14.266098 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c76b7925-1d11-48fa-82eb-53b5aa304bb3 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:11:14.280715 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application [None req-c76b7925-1d11-48fa-82eb-53b5aa304bb3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 12:11:14.280715 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:11:14.280715 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:11:14.280715 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:11:14.280715 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:11:14.280715 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:11:14.280715 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:11:14.280715 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:11:14.280715 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:11:14.280715 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:11:14.280715 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:11:14.280715 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:11:14.280715 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 12:11:14.280715 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 12:11:14.280715 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 12:11:14.280715 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 12:11:14.281645 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:11:14.281645 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:11:14.281645 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:11:14.281645 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:11:14.281645 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:11:14.281645 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:11:14.281645 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:11:14.281645 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application with Lock( Sep 15 12:11:14.281645 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:11:14.281645 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:11:14.281645 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:11:14.281645 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:11:14.281645 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:11:14.281645 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:11:14.281645 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:11:14.281645 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:11:14.281645 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 12:11:14.282501 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 12:11:14.282501 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 12:11:14.282501 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 15 12:11:14.282501 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 253/501] 10.222.0.15 () {56 vars in 1108 bytes} [Thu Sep 15 12:11:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:11:14.282724 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76b7925-1d11-48fa-82eb-53b5aa304bb3 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:14.289959 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9400ec27-054c-4647-a163-2eef957b18ae None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:14.322987 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e204ec-52d2-44d0-a52a-d7895867dc5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:14.323614 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e204ec-52d2-44d0-a52a-d7895867dc5a None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:14.323893 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e204ec-52d2-44d0-a52a-d7895867dc5a None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:14.324165 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e204ec-52d2-44d0-a52a-d7895867dc5a None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:14.331268 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e204ec-52d2-44d0-a52a-d7895867dc5a None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:11:14.332698 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e204ec-52d2-44d0-a52a-d7895867dc5a None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:11:14.345327 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application [None req-c9e204ec-52d2-44d0-a52a-d7895867dc5a None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 15 12:11:14.345327 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:11:14.345327 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:11:14.345327 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:11:14.345327 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:11:14.345327 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:11:14.345327 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:11:14.345327 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:11:14.345327 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:11:14.345327 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:11:14.345327 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:11:14.345327 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:11:14.345327 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 15 12:11:14.345327 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 15 12:11:14.345327 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 15 12:11:14.345327 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 15 12:11:14.346266 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:11:14.346266 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:11:14.346266 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 15 12:11:14.346266 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 12:11:14.346266 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 15 12:11:14.346266 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 12:11:14.346266 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:11:14.346266 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:11:14.346266 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:11:14.346266 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:11:14.346266 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:11:14.346266 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application with Lock( Sep 15 12:11:14.346266 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:11:14.346266 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:11:14.346266 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:11:14.346266 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:11:14.346266 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:11:14.346266 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:11:14.347162 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:11:14.347162 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:11:14.347162 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 15 12:11:14.347162 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 15 12:11:14.347162 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 15 12:11:14.347162 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 15 12:11:14.347162 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 15 12:11:14.347162 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 15 12:11:14.347162 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 15 12:11:14.347162 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application Sep 15 12:11:14.347162 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 249/502] 10.222.0.15 () {56 vars in 1099 bytes} [Thu Sep 15 12:11:14 2022] GET /identity/v3/users/cinder => generated 84 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:11:14.347812 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2312f1-0c70-4b6f-b676-e5cdd09e8894 None None] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:14.347812 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2312f1-0c70-4b6f-b676-e5cdd09e8894 None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:14.348004 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2312f1-0c70-4b6f-b676-e5cdd09e8894 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:14.365391 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.auth.core [None req-ae2312f1-0c70-4b6f-b676-e5cdd09e8894 None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81449) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:11:14.366572 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-ae2312f1-0c70-4b6f-b676-e5cdd09e8894 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:11:14.383922 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76b7925-1d11-48fa-82eb-53b5aa304bb3 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:14.405646 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:11:14.417439 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd8d459-0ce5-4aec-9932-ce4b186d6418 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:14.418040 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd8d459-0ce5-4aec-9932-ce4b186d6418 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:14.418321 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd8d459-0ce5-4aec-9932-ce4b186d6418 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:14.418592 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd8d459-0ce5-4aec-9932-ce4b186d6418 None admin] PATH_INFO: `/v3/roles` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:14.419245 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd8d459-0ce5-4aec-9932-ce4b186d6418 None admin] RBAC: Adding query filter params (name=service) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:11:14.419552 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd8d459-0ce5-4aec-9932-ce4b186d6418 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:11:14.420931 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd8d459-0ce5-4aec-9932-ce4b186d6418 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:11:14.429196 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 254/503] 10.222.0.15 () {56 vars in 1103 bytes} [Thu Sep 15 12:11:14 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:11:14.430102 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd8d459-0ce5-4aec-9932-ce4b186d6418 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:14.437060 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 250/504] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 12:11:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 12:11:14.437060 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2312f1-0c70-4b6f-b676-e5cdd09e8894 None None] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:14.443988 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:11:14.538936 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2312f1-0c70-4b6f-b676-e5cdd09e8894 None None] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:14.540870 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd8d459-0ce5-4aec-9932-ce4b186d6418 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:14.572410 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a2da48-af08-4853-a6a4-28a602168831 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:14.573102 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a2da48-af08-4853-a6a4-28a602168831 None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:14.573335 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a2da48-af08-4853-a6a4-28a602168831 None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:14.573596 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a2da48-af08-4853-a6a4-28a602168831 None admin] PATH_INFO: `/v3/domains` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:14.574235 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a2da48-af08-4853-a6a4-28a602168831 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81449) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:11:14.574655 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a2da48-af08-4853-a6a4-28a602168831 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:11:14.576243 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a2da48-af08-4853-a6a4-28a602168831 None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:11:14.581028 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096cc943-2c8a-457d-9825-8940e4ae56b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:14.585161 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096cc943-2c8a-457d-9825-8940e4ae56b0 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:14.585624 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096cc943-2c8a-457d-9825-8940e4ae56b0 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:14.586092 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096cc943-2c8a-457d-9825-8940e4ae56b0 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:14.590701 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 251/505] 10.222.0.15 () {56 vars in 1109 bytes} [Thu Sep 15 12:11:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:11:14.591386 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a2da48-af08-4853-a6a4-28a602168831 None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:14.602236 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096cc943-2c8a-457d-9825-8940e4ae56b0 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:11:14.603886 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096cc943-2c8a-457d-9825-8940e4ae56b0 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:11:14.620932 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application [None req-096cc943-2c8a-457d-9825-8940e4ae56b0 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 15 12:11:14.620932 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:11:14.620932 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:11:14.620932 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:11:14.620932 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:11:14.620932 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:11:14.620932 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:11:14.620932 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:11:14.620932 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:11:14.620932 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:11:14.620932 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:11:14.620932 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:11:14.620932 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 15 12:11:14.620932 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 15 12:11:14.620932 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 15 12:11:14.620932 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 15 12:11:14.622334 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:11:14.622334 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:11:14.622334 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 15 12:11:14.622334 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 12:11:14.622334 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 15 12:11:14.622334 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 12:11:14.622334 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:11:14.622334 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:11:14.622334 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:11:14.622334 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:11:14.622334 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:11:14.622334 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application with Lock( Sep 15 12:11:14.622334 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:11:14.622334 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:11:14.622334 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:11:14.622334 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:11:14.622334 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:11:14.622334 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:11:14.623878 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:11:14.623878 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:11:14.623878 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 15 12:11:14.623878 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 15 12:11:14.623878 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 15 12:11:14.623878 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 15 12:11:14.623878 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 15 12:11:14.623878 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 15 12:11:14.623878 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 15 12:11:14.623878 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 15 12:11:14.626411 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 255/506] 10.222.0.15 () {56 vars in 1093 bytes} [Thu Sep 15 12:11:14 2022] GET /identity/v3/users/nova => generated 82 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:11:14.627293 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096cc943-2c8a-457d-9825-8940e4ae56b0 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:14.666145 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:11:14.679624 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:11:14.700798 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a2da48-af08-4853-a6a4-28a602168831 None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:14.735593 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0b43ca-5a76-4682-ab4d-14d1f514d3b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:14.736206 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0b43ca-5a76-4682-ab4d-14d1f514d3b3 None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:14.736485 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0b43ca-5a76-4682-ab4d-14d1f514d3b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:14.736781 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0b43ca-5a76-4682-ab4d-14d1f514d3b3 None admin] PATH_INFO: `/v3/users` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:14.737095 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096cc943-2c8a-457d-9825-8940e4ae56b0 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:14.737508 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0b43ca-5a76-4682-ab4d-14d1f514d3b3 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81449) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:11:14.737829 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0b43ca-5a76-4682-ab4d-14d1f514d3b3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:11:14.739256 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0b43ca-5a76-4682-ab4d-14d1f514d3b3 None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:11:14.769683 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fbdc6fb-95f1-475b-8b43-f67c384b8727 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:14.770315 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbdc6fb-95f1-475b-8b43-f67c384b8727 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:14.770596 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbdc6fb-95f1-475b-8b43-f67c384b8727 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:14.770869 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbdc6fb-95f1-475b-8b43-f67c384b8727 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:14.776809 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 252/507] 10.222.0.15 () {56 vars in 1137 bytes} [Thu Sep 15 12:11:14 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:11:14.777625 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0b43ca-5a76-4682-ab4d-14d1f514d3b3 None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:14.782815 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fbdc6fb-95f1-475b-8b43-f67c384b8727 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:11:14.784269 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fbdc6fb-95f1-475b-8b43-f67c384b8727 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:11:14.798265 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application [None req-7fbdc6fb-95f1-475b-8b43-f67c384b8727 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 12:11:14.798265 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:11:14.798265 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:11:14.798265 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:11:14.798265 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:11:14.798265 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:11:14.798265 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:11:14.798265 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:11:14.798265 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:11:14.798265 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:11:14.798265 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:11:14.798265 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:11:14.798265 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 12:11:14.798265 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 12:11:14.798265 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 12:11:14.798265 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 12:11:14.799317 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:11:14.799317 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:11:14.799317 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:11:14.799317 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:11:14.799317 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:11:14.799317 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:11:14.799317 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:11:14.799317 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application with Lock( Sep 15 12:11:14.799317 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:11:14.799317 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:11:14.799317 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:11:14.799317 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:11:14.799317 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:11:14.799317 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:11:14.799317 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:11:14.799317 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:11:14.799317 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 12:11:14.800234 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 12:11:14.800234 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 12:11:14.800234 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 15 12:11:14.800234 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 256/508] 10.222.0.15 () {56 vars in 1108 bytes} [Thu Sep 15 12:11:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:11:14.800234 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fbdc6fb-95f1-475b-8b43-f67c384b8727 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:14.876658 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0b43ca-5a76-4682-ab4d-14d1f514d3b3 None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:14.892396 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:11:14.899351 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fbdc6fb-95f1-475b-8b43-f67c384b8727 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:14.902387 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:11:14.911178 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7966d567-7ec8-4a0f-902e-12089fc7193d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:14.911812 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7966d567-7ec8-4a0f-902e-12089fc7193d None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:14.912093 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7966d567-7ec8-4a0f-902e-12089fc7193d None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:14.912368 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7966d567-7ec8-4a0f-902e-12089fc7193d None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:14.919267 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7966d567-7ec8-4a0f-902e-12089fc7193d None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:11:14.920720 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7966d567-7ec8-4a0f-902e-12089fc7193d None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:11:14.932216 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application [None req-7966d567-7ec8-4a0f-902e-12089fc7193d None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 15 12:11:14.932216 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:11:14.932216 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:11:14.932216 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:11:14.932216 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:11:14.932216 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:11:14.932216 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:11:14.932216 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:11:14.932216 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:11:14.932216 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:11:14.932216 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:11:14.932216 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:11:14.932216 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 15 12:11:14.932216 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 15 12:11:14.932216 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 15 12:11:14.932216 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 15 12:11:14.933155 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:11:14.933155 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:11:14.933155 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 15 12:11:14.933155 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 12:11:14.933155 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 15 12:11:14.933155 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 12:11:14.933155 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:11:14.933155 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:11:14.933155 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:11:14.933155 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:11:14.933155 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:11:14.933155 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application with Lock( Sep 15 12:11:14.933155 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:11:14.933155 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:11:14.933155 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:11:14.933155 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:11:14.933155 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:11:14.933155 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:11:14.934054 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:11:14.934054 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:11:14.934054 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 15 12:11:14.934054 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 15 12:11:14.934054 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 15 12:11:14.934054 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 15 12:11:14.934054 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 15 12:11:14.934054 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 15 12:11:14.934054 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 15 12:11:14.934054 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application Sep 15 12:11:14.934567 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052a50f5-3cf8-4885-9223-3b974aadd372 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:14.934567 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052a50f5-3cf8-4885-9223-3b974aadd372 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:14.934567 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052a50f5-3cf8-4885-9223-3b974aadd372 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:14.934763 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 253/509] 10.222.0.15 () {56 vars in 1102 bytes} [Thu Sep 15 12:11:14 2022] GET /identity/v3/users/neutron => generated 85 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:11:14.934829 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052a50f5-3cf8-4885-9223-3b974aadd372 None admin] PATH_INFO: `/v3/users` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:14.934894 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7966d567-7ec8-4a0f-902e-12089fc7193d None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:14.934959 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052a50f5-3cf8-4885-9223-3b974aadd372 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:11:14.934959 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052a50f5-3cf8-4885-9223-3b974aadd372 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:11:14.935963 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052a50f5-3cf8-4885-9223-3b974aadd372 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:11:14.975750 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 257/510] 10.222.0.15 () {56 vars in 1133 bytes} [Thu Sep 15 12:11:14 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:11:14.976616 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052a50f5-3cf8-4885-9223-3b974aadd372 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:15.036557 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7966d567-7ec8-4a0f-902e-12089fc7193d None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:15.070018 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34c9f3f-a324-493e-a78f-22c74d4f175e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:15.070581 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34c9f3f-a324-493e-a78f-22c74d4f175e None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:15.070863 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34c9f3f-a324-493e-a78f-22c74d4f175e None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:15.071142 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34c9f3f-a324-493e-a78f-22c74d4f175e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:15.076559 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052a50f5-3cf8-4885-9223-3b974aadd372 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:15.081467 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34c9f3f-a324-493e-a78f-22c74d4f175e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:11:15.082932 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34c9f3f-a324-493e-a78f-22c74d4f175e None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:11:15.097386 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application [None req-a34c9f3f-a324-493e-a78f-22c74d4f175e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 12:11:15.097386 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:11:15.097386 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:11:15.097386 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:11:15.097386 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:11:15.097386 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:11:15.097386 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:11:15.097386 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:11:15.097386 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:11:15.097386 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:11:15.097386 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:11:15.097386 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:11:15.097386 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 12:11:15.097386 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 12:11:15.097386 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 12:11:15.097386 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 12:11:15.098473 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:11:15.098473 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:11:15.098473 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:11:15.098473 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:11:15.098473 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:11:15.098473 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:11:15.098473 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:11:15.098473 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application with Lock( Sep 15 12:11:15.098473 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:11:15.098473 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:11:15.098473 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:11:15.098473 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:11:15.098473 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:11:15.098473 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:11:15.098473 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:11:15.098473 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:11:15.098473 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 12:11:15.099316 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 12:11:15.099316 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 12:11:15.099316 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application Sep 15 12:11:15.099316 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 254/511] 10.222.0.15 () {56 vars in 1108 bytes} [Thu Sep 15 12:11:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:11:15.099316 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34c9f3f-a324-493e-a78f-22c74d4f175e None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:15.112626 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974098c2-0013-4796-a35b-ed7332f23331 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:15.113269 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974098c2-0013-4796-a35b-ed7332f23331 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:15.113551 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974098c2-0013-4796-a35b-ed7332f23331 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:15.113833 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974098c2-0013-4796-a35b-ed7332f23331 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:15.138278 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:11:15.139733 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:11:15.183890 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-974098c2-0013-4796-a35b-ed7332f23331 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:11:15.185383 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-974098c2-0013-4796-a35b-ed7332f23331 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:11:15.199109 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34c9f3f-a324-493e-a78f-22c74d4f175e None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:15.232613 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8801c2e-9fe9-4804-b05e-9b818f0c176d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:15.233269 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8801c2e-9fe9-4804-b05e-9b818f0c176d None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:15.233545 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8801c2e-9fe9-4804-b05e-9b818f0c176d None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:15.233813 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8801c2e-9fe9-4804-b05e-9b818f0c176d None admin] PATH_INFO: `/v3/domains` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:15.234442 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8801c2e-9fe9-4804-b05e-9b818f0c176d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81449) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:11:15.234739 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8801c2e-9fe9-4804-b05e-9b818f0c176d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:11:15.236091 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8801c2e-9fe9-4804-b05e-9b818f0c176d None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:11:15.246825 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 255/512] 10.222.0.15 () {56 vars in 1109 bytes} [Thu Sep 15 12:11:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:11:15.247579 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8801c2e-9fe9-4804-b05e-9b818f0c176d None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:15.283593 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 258/513] 10.222.0.15 () {58 vars in 1282 bytes} [Thu Sep 15 12:11:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 15 12:11:15.284428 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974098c2-0013-4796-a35b-ed7332f23331 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:15.337842 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:11:15.343808 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:11:15.347988 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8801c2e-9fe9-4804-b05e-9b818f0c176d None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:15.382760 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29efa945-0774-4711-9b09-4c35eb0682fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:15.383371 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-974098c2-0013-4796-a35b-ed7332f23331 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:15.383501 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29efa945-0774-4711-9b09-4c35eb0682fd None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:15.383658 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29efa945-0774-4711-9b09-4c35eb0682fd None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:15.383921 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29efa945-0774-4711-9b09-4c35eb0682fd None admin] PATH_INFO: `/v3/users` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:15.384785 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29efa945-0774-4711-9b09-4c35eb0682fd None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81449) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:11:15.384969 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29efa945-0774-4711-9b09-4c35eb0682fd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:11:15.386393 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29efa945-0774-4711-9b09-4c35eb0682fd None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:11:15.404171 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 256/514] 10.222.0.15 () {56 vars in 1139 bytes} [Thu Sep 15 12:11:15 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:11:15.404992 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29efa945-0774-4711-9b09-4c35eb0682fd None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:15.418103 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce0f02f-244d-432d-b9e6-54bfb9144ad8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:15.418716 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce0f02f-244d-432d-b9e6-54bfb9144ad8 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:15.418998 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce0f02f-244d-432d-b9e6-54bfb9144ad8 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:15.419271 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce0f02f-244d-432d-b9e6-54bfb9144ad8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:15.486469 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce0f02f-244d-432d-b9e6-54bfb9144ad8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:11:15.487926 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce0f02f-244d-432d-b9e6-54bfb9144ad8 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:11:15.503701 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29efa945-0774-4711-9b09-4c35eb0682fd None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:15.538968 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24a598b-2165-471f-9ba3-928d50587f33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:15.539578 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24a598b-2165-471f-9ba3-928d50587f33 None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:15.539864 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24a598b-2165-471f-9ba3-928d50587f33 None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:15.540135 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24a598b-2165-471f-9ba3-928d50587f33 None admin] PATH_INFO: `/v3/domains` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:15.540776 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24a598b-2165-471f-9ba3-928d50587f33 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81449) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:11:15.541102 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24a598b-2165-471f-9ba3-928d50587f33 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:11:15.542484 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e24a598b-2165-471f-9ba3-928d50587f33 None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:11:15.542623 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:11:15.554396 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 257/515] 10.222.0.15 () {56 vars in 1109 bytes} [Thu Sep 15 12:11:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:11:15.555186 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24a598b-2165-471f-9ba3-928d50587f33 None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:15.572628 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:11:15.596878 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 259/516] 10.222.0.15 () {58 vars in 1282 bytes} [Thu Sep 15 12:11:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 15 12:11:15.597735 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce0f02f-244d-432d-b9e6-54bfb9144ad8 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:15.658146 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24a598b-2165-471f-9ba3-928d50587f33 None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:15.691522 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d860e823-4034-4385-b98e-ea1c5aa45183 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:15.692157 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d860e823-4034-4385-b98e-ea1c5aa45183 None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:15.692438 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d860e823-4034-4385-b98e-ea1c5aa45183 None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:15.692706 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d860e823-4034-4385-b98e-ea1c5aa45183 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:15.694937 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce0f02f-244d-432d-b9e6-54bfb9144ad8 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:15.699941 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d860e823-4034-4385-b98e-ea1c5aa45183 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:11:15.701405 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d860e823-4034-4385-b98e-ea1c5aa45183 None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:11:15.712200 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application [None req-d860e823-4034-4385-b98e-ea1c5aa45183 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 15 12:11:15.712200 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:11:15.712200 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:11:15.712200 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:11:15.712200 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:11:15.712200 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:11:15.712200 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:11:15.712200 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:11:15.712200 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:11:15.712200 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:11:15.712200 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:11:15.712200 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:11:15.712200 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 15 12:11:15.712200 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 15 12:11:15.712200 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 15 12:11:15.712200 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 15 12:11:15.713239 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:11:15.713239 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:11:15.713239 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 15 12:11:15.713239 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 12:11:15.713239 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 15 12:11:15.713239 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 12:11:15.713239 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:11:15.713239 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:11:15.713239 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:11:15.713239 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:11:15.713239 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:11:15.713239 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application with Lock( Sep 15 12:11:15.713239 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:11:15.713239 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:11:15.713239 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:11:15.713239 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:11:15.713239 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:11:15.713239 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:11:15.714285 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:11:15.714285 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:11:15.714285 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 15 12:11:15.714285 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 15 12:11:15.714285 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 15 12:11:15.714285 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 15 12:11:15.714285 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 15 12:11:15.714285 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 15 12:11:15.714285 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 15 12:11:15.714285 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application Sep 15 12:11:15.714285 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 258/517] 10.222.0.15 () {56 vars in 1099 bytes} [Thu Sep 15 12:11:15 2022] GET /identity/v3/users/glance => generated 84 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:11:15.714285 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d860e823-4034-4385-b98e-ea1c5aa45183 None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:15.732476 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ce8f79-961a-452c-9f14-235a10ef53d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:15.733151 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ce8f79-961a-452c-9f14-235a10ef53d0 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:15.733427 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ce8f79-961a-452c-9f14-235a10ef53d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:15.733694 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ce8f79-961a-452c-9f14-235a10ef53d0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:15.744799 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ce8f79-961a-452c-9f14-235a10ef53d0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:11:15.746231 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ce8f79-961a-452c-9f14-235a10ef53d0 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:11:15.760465 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application [None req-f1ce8f79-961a-452c-9f14-235a10ef53d0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 12:11:15.760465 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:11:15.760465 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:11:15.760465 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:11:15.760465 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:11:15.760465 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:11:15.760465 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:11:15.760465 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:11:15.760465 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:11:15.760465 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:11:15.760465 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:11:15.760465 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:11:15.760465 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 12:11:15.760465 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 12:11:15.760465 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 12:11:15.760465 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 12:11:15.761531 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:11:15.761531 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:11:15.761531 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:11:15.761531 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:11:15.761531 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:11:15.761531 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:11:15.761531 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:11:15.761531 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application with Lock( Sep 15 12:11:15.761531 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:11:15.761531 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:11:15.761531 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:11:15.761531 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:11:15.761531 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:11:15.761531 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:11:15.761531 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:11:15.761531 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:11:15.761531 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 12:11:15.762560 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 12:11:15.762560 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 12:11:15.762560 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 15 12:11:15.762560 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 260/518] 10.222.0.15 () {56 vars in 1108 bytes} [Thu Sep 15 12:11:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:11:15.762816 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ce8f79-961a-452c-9f14-235a10ef53d0 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:15.771258 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:11:15.793275 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:11:15.817001 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d860e823-4034-4385-b98e-ea1c5aa45183 None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:15.850420 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bb3e07-f3c3-45b6-8419-8bdf236d6045 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:15.850960 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bb3e07-f3c3-45b6-8419-8bdf236d6045 None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:15.851241 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bb3e07-f3c3-45b6-8419-8bdf236d6045 None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:15.851512 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bb3e07-f3c3-45b6-8419-8bdf236d6045 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:15.861695 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ce8f79-961a-452c-9f14-235a10ef53d0 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:15.895848 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc88c6c0-d77d-4b63-9d83-ef0b6d1cb007 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:15.896998 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc88c6c0-d77d-4b63-9d83-ef0b6d1cb007 None admin] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:15.897276 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc88c6c0-d77d-4b63-9d83-ef0b6d1cb007 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:15.897567 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc88c6c0-d77d-4b63-9d83-ef0b6d1cb007 None admin] PATH_INFO: `/v3/users` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:15.898199 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc88c6c0-d77d-4b63-9d83-ef0b6d1cb007 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:11:15.899623 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc88c6c0-d77d-4b63-9d83-ef0b6d1cb007 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:11:15.920747 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05bb3e07-f3c3-45b6-8419-8bdf236d6045 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:11:15.921921 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 261/519] 10.222.0.15 () {60 vars in 1132 bytes} [Thu Sep 15 12:11:15 2022] POST /identity/v3/users => generated 247 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 12:11:15.922243 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05bb3e07-f3c3-45b6-8419-8bdf236d6045 None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:11:15.922851 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc88c6c0-d77d-4b63-9d83-ef0b6d1cb007 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:16.011284 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:11:16.021867 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:11:16.058556 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc88c6c0-d77d-4b63-9d83-ef0b6d1cb007 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:16.058754 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 259/520] 10.222.0.15 () {58 vars in 1282 bytes} [Thu Sep 15 12:11:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 15 12:11:16.058754 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bb3e07-f3c3-45b6-8419-8bdf236d6045 None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:16.098907 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36005d1a-0fb7-4f67-a874-cdfaf910376d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:16.100112 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36005d1a-0fb7-4f67-a874-cdfaf910376d None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:16.100400 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36005d1a-0fb7-4f67-a874-cdfaf910376d None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:16.100724 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36005d1a-0fb7-4f67-a874-cdfaf910376d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:16.113933 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36005d1a-0fb7-4f67-a874-cdfaf910376d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:11:16.115429 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36005d1a-0fb7-4f67-a874-cdfaf910376d None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:11:16.130572 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application [None req-36005d1a-0fb7-4f67-a874-cdfaf910376d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 12:11:16.130572 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:11:16.130572 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:11:16.130572 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:11:16.130572 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:11:16.130572 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:11:16.130572 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:11:16.130572 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:11:16.130572 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:11:16.130572 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:11:16.130572 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:11:16.130572 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:11:16.130572 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 12:11:16.130572 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 12:11:16.130572 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 12:11:16.130572 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 12:11:16.131558 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:11:16.131558 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:11:16.131558 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:11:16.131558 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:11:16.131558 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:11:16.131558 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:11:16.131558 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:11:16.131558 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application with Lock( Sep 15 12:11:16.131558 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:11:16.131558 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:11:16.131558 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:11:16.131558 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:11:16.131558 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:11:16.131558 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:11:16.131558 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:11:16.131558 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:11:16.131558 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 12:11:16.132577 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 12:11:16.132577 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 12:11:16.132577 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 15 12:11:16.132577 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 262/521] 10.222.0.15 () {56 vars in 1108 bytes} [Thu Sep 15 12:11:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:11:16.132577 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36005d1a-0fb7-4f67-a874-cdfaf910376d None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:16.173744 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bb3e07-f3c3-45b6-8419-8bdf236d6045 None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:16.210026 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492ba755-0943-45f9-aa8b-a0c7f1bb925b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:16.210633 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492ba755-0943-45f9-aa8b-a0c7f1bb925b None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:16.210915 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492ba755-0943-45f9-aa8b-a0c7f1bb925b None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:16.211186 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492ba755-0943-45f9-aa8b-a0c7f1bb925b None admin] PATH_INFO: `/v3/users` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:16.211907 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-492ba755-0943-45f9-aa8b-a0c7f1bb925b None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81449) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:11:16.212221 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-492ba755-0943-45f9-aa8b-a0c7f1bb925b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:11:16.213676 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-492ba755-0943-45f9-aa8b-a0c7f1bb925b None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:11:16.232883 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 260/522] 10.222.0.15 () {56 vars in 1137 bytes} [Thu Sep 15 12:11:16 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:11:16.232883 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492ba755-0943-45f9-aa8b-a0c7f1bb925b None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:16.256123 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36005d1a-0fb7-4f67-a874-cdfaf910376d None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:16.267708 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:11:16.276818 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:11:16.300703 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7555f437-92d2-4a2c-9dc0-69953f065cdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:16.301344 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7555f437-92d2-4a2c-9dc0-69953f065cdf None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:16.301625 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7555f437-92d2-4a2c-9dc0-69953f065cdf None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:16.301916 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7555f437-92d2-4a2c-9dc0-69953f065cdf None admin] PATH_INFO: `/v3/domains` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:16.303088 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7555f437-92d2-4a2c-9dc0-69953f065cdf None admin] RBAC: Adding query filter params (name=Default) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:11:16.303387 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7555f437-92d2-4a2c-9dc0-69953f065cdf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:11:16.304748 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7555f437-92d2-4a2c-9dc0-69953f065cdf None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:11:16.316472 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 263/523] 10.222.0.15 () {56 vars in 1109 bytes} [Thu Sep 15 12:11:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:11:16.317323 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7555f437-92d2-4a2c-9dc0-69953f065cdf None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:16.345368 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492ba755-0943-45f9-aa8b-a0c7f1bb925b None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:16.381313 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda73e0d-c629-415e-a9ef-bca5c92ddd98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:16.382193 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda73e0d-c629-415e-a9ef-bca5c92ddd98 None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:16.382659 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda73e0d-c629-415e-a9ef-bca5c92ddd98 None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:16.382992 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda73e0d-c629-415e-a9ef-bca5c92ddd98 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:16.393290 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda73e0d-c629-415e-a9ef-bca5c92ddd98 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:11:16.394732 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda73e0d-c629-415e-a9ef-bca5c92ddd98 None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:11:16.409970 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application [None req-eda73e0d-c629-415e-a9ef-bca5c92ddd98 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 12:11:16.409970 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:11:16.409970 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:11:16.409970 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:11:16.409970 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:11:16.409970 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:11:16.409970 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:11:16.409970 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:11:16.409970 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:11:16.409970 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:11:16.409970 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:11:16.409970 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:11:16.409970 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 12:11:16.409970 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 12:11:16.409970 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 12:11:16.409970 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 12:11:16.410931 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:11:16.410931 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:11:16.410931 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:11:16.410931 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:11:16.410931 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:11:16.410931 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:11:16.410931 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:11:16.410931 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application with Lock( Sep 15 12:11:16.410931 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:11:16.410931 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:11:16.410931 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:11:16.410931 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:11:16.410931 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:11:16.410931 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:11:16.410931 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:11:16.410931 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:11:16.410931 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 12:11:16.412341 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 12:11:16.412341 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 12:11:16.412341 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application Sep 15 12:11:16.412341 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 261/524] 10.222.0.15 () {56 vars in 1108 bytes} [Thu Sep 15 12:11:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:11:16.412341 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda73e0d-c629-415e-a9ef-bca5c92ddd98 None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:16.425670 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7555f437-92d2-4a2c-9dc0-69953f065cdf None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:16.460445 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c355d4-6241-4800-8858-64303e6bc923 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:16.461115 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c355d4-6241-4800-8858-64303e6bc923 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:16.461392 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c355d4-6241-4800-8858-64303e6bc923 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:16.461658 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c355d4-6241-4800-8858-64303e6bc923 None admin] PATH_INFO: `/v3/domains` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:16.462290 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c355d4-6241-4800-8858-64303e6bc923 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:11:16.462590 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c355d4-6241-4800-8858-64303e6bc923 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:11:16.463941 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c355d4-6241-4800-8858-64303e6bc923 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:11:16.475994 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 264/525] 10.222.0.15 () {56 vars in 1109 bytes} [Thu Sep 15 12:11:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:11:16.481201 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c355d4-6241-4800-8858-64303e6bc923 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:16.501955 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:11:16.517435 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda73e0d-c629-415e-a9ef-bca5c92ddd98 None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:16.518790 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:11:16.554254 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb42e2b-0f6a-4486-a252-2f9014ea1369 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:16.555078 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb42e2b-0f6a-4486-a252-2f9014ea1369 None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:16.555357 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb42e2b-0f6a-4486-a252-2f9014ea1369 None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:16.555626 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb42e2b-0f6a-4486-a252-2f9014ea1369 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:16.593418 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c355d4-6241-4800-8858-64303e6bc923 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:16.625718 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb42e2b-0f6a-4486-a252-2f9014ea1369 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:11:16.627233 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb42e2b-0f6a-4486-a252-2f9014ea1369 None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:11:16.631793 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922cae0c-c50e-4928-b1c9-d0bc40c0ade5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:16.632382 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922cae0c-c50e-4928-b1c9-d0bc40c0ade5 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:16.632662 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922cae0c-c50e-4928-b1c9-d0bc40c0ade5 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:16.632957 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922cae0c-c50e-4928-b1c9-d0bc40c0ade5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:16.640094 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922cae0c-c50e-4928-b1c9-d0bc40c0ade5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:11:16.641538 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922cae0c-c50e-4928-b1c9-d0bc40c0ade5 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:11:16.654329 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application [None req-922cae0c-c50e-4928-b1c9-d0bc40c0ade5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 12:11:16.654329 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:11:16.654329 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:11:16.654329 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:11:16.654329 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:11:16.654329 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:11:16.654329 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:11:16.654329 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:11:16.654329 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:11:16.654329 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:11:16.654329 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:11:16.654329 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:11:16.654329 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 15 12:11:16.654329 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 15 12:11:16.654329 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 15 12:11:16.655229 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 15 12:11:16.655229 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:11:16.655229 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:11:16.655229 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:11:16.655229 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:11:16.655229 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:11:16.655229 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:11:16.655229 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:11:16.655229 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application with Lock( Sep 15 12:11:16.655229 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:11:16.655229 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:11:16.655229 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:11:16.655229 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:11:16.655229 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:11:16.655229 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:11:16.655229 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:11:16.655229 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:11:16.656118 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 15 12:11:16.656118 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 15 12:11:16.656118 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 15 12:11:16.656118 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 15 12:11:16.656118 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 15 12:11:16.656118 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 15 12:11:16.656118 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 12:11:16.656118 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 15 12:11:16.656118 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 265/526] 10.222.0.15 () {56 vars in 1111 bytes} [Thu Sep 15 12:11:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:11:16.656698 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922cae0c-c50e-4928-b1c9-d0bc40c0ade5 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:16.721735 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:11:16.743886 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 262/527] 10.222.0.15 () {58 vars in 1282 bytes} [Thu Sep 15 12:11:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 15 12:11:16.743886 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb42e2b-0f6a-4486-a252-2f9014ea1369 None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:16.760290 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:11:16.767562 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922cae0c-c50e-4928-b1c9-d0bc40c0ade5 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:16.804105 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89a3d83-3454-4141-8e9d-530ee4e0524b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:16.804834 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89a3d83-3454-4141-8e9d-530ee4e0524b None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:16.805183 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89a3d83-3454-4141-8e9d-530ee4e0524b None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:16.805464 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89a3d83-3454-4141-8e9d-530ee4e0524b None admin] PATH_INFO: `/v3/domains` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:16.806106 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e89a3d83-3454-4141-8e9d-530ee4e0524b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:11:16.806411 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e89a3d83-3454-4141-8e9d-530ee4e0524b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:11:16.807829 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e89a3d83-3454-4141-8e9d-530ee4e0524b None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:11:16.819808 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 266/528] 10.222.0.15 () {56 vars in 1109 bytes} [Thu Sep 15 12:11:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:11:16.820618 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89a3d83-3454-4141-8e9d-530ee4e0524b None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:16.850341 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb42e2b-0f6a-4486-a252-2f9014ea1369 None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:16.887310 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6009c989-e1d9-4d08-9bba-aa942c00a17e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:16.887856 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6009c989-e1d9-4d08-9bba-aa942c00a17e None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:16.888133 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6009c989-e1d9-4d08-9bba-aa942c00a17e None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:16.888406 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6009c989-e1d9-4d08-9bba-aa942c00a17e None admin] PATH_INFO: `/v3/projects/service` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:16.895185 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6009c989-e1d9-4d08-9bba-aa942c00a17e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:11:16.897156 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6009c989-e1d9-4d08-9bba-aa942c00a17e None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:11:16.909351 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application [None req-6009c989-e1d9-4d08-9bba-aa942c00a17e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 12:11:16.909351 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:11:16.909351 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:11:16.909351 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:11:16.909351 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:11:16.909351 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:11:16.909351 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:11:16.909351 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:11:16.909351 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:11:16.909351 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:11:16.909351 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:11:16.909351 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:11:16.909351 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 15 12:11:16.909351 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 15 12:11:16.909351 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 15 12:11:16.910333 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 15 12:11:16.910333 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:11:16.910333 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:11:16.910333 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:11:16.910333 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:11:16.910333 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:11:16.910333 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:11:16.910333 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:11:16.910333 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application with Lock( Sep 15 12:11:16.910333 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:11:16.910333 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:11:16.910333 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:11:16.910333 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:11:16.910333 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:11:16.910333 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:11:16.910333 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:11:16.910333 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:11:16.911352 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 15 12:11:16.911352 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 15 12:11:16.911352 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 15 12:11:16.911352 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 15 12:11:16.911352 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 15 12:11:16.911352 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 15 12:11:16.911352 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 12:11:16.911352 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application Sep 15 12:11:16.911352 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 263/529] 10.222.0.15 () {56 vars in 1111 bytes} [Thu Sep 15 12:11:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:11:16.911352 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6009c989-e1d9-4d08-9bba-aa942c00a17e None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:16.936548 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89a3d83-3454-4141-8e9d-530ee4e0524b None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:16.968689 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:11:16.977611 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec358f26-47bc-43c6-86d8-ef91190b8c67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:16.978231 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec358f26-47bc-43c6-86d8-ef91190b8c67 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:16.978655 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec358f26-47bc-43c6-86d8-ef91190b8c67 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:16.978922 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec358f26-47bc-43c6-86d8-ef91190b8c67 None admin] PATH_INFO: `/v3/projects` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:16.979613 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec358f26-47bc-43c6-86d8-ef91190b8c67 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:11:16.980328 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec358f26-47bc-43c6-86d8-ef91190b8c67 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:11:16.982114 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec358f26-47bc-43c6-86d8-ef91190b8c67 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:11:17.000465 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 267/530] 10.222.0.15 () {56 vars in 1148 bytes} [Thu Sep 15 12:11:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:11:17.001226 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec358f26-47bc-43c6-86d8-ef91190b8c67 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:17.017984 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:11:17.034561 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6009c989-e1d9-4d08-9bba-aa942c00a17e None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:17.071072 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d464f7-70fd-4f25-9e90-6b8a3a21b8af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:17.071710 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d464f7-70fd-4f25-9e90-6b8a3a21b8af None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:17.071987 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d464f7-70fd-4f25-9e90-6b8a3a21b8af None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:17.072258 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d464f7-70fd-4f25-9e90-6b8a3a21b8af None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:17.083073 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d464f7-70fd-4f25-9e90-6b8a3a21b8af None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:11:17.085596 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91d464f7-70fd-4f25-9e90-6b8a3a21b8af None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:11:17.100449 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application [None req-91d464f7-70fd-4f25-9e90-6b8a3a21b8af None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 12:11:17.100449 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:11:17.100449 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:11:17.100449 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:11:17.100449 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:11:17.100449 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:11:17.100449 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:11:17.100449 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:11:17.100449 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:11:17.100449 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:11:17.100449 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:11:17.100449 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:11:17.100449 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 12:11:17.100449 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 12:11:17.100449 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 12:11:17.100449 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 12:11:17.101532 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:11:17.101532 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:11:17.101532 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:11:17.101532 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:11:17.101532 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:11:17.101532 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:11:17.101532 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:11:17.101532 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application with Lock( Sep 15 12:11:17.101532 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:11:17.101532 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:11:17.101532 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:11:17.101532 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:11:17.101532 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:11:17.101532 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:11:17.101532 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:11:17.101532 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:11:17.101532 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 12:11:17.102425 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 12:11:17.102425 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 12:11:17.102425 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application Sep 15 12:11:17.102425 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 264/531] 10.222.0.15 () {56 vars in 1108 bytes} [Thu Sep 15 12:11:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:11:17.102684 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d464f7-70fd-4f25-9e90-6b8a3a21b8af None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:17.115885 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec358f26-47bc-43c6-86d8-ef91190b8c67 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:17.154355 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d1015c-c37e-4972-9273-56deb6d14bab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:17.154896 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d1015c-c37e-4972-9273-56deb6d14bab None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:17.155170 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d1015c-c37e-4972-9273-56deb6d14bab None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:17.155592 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d1015c-c37e-4972-9273-56deb6d14bab None admin] PATH_INFO: `/v3/projects/service` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:17.163576 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d1015c-c37e-4972-9273-56deb6d14bab None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:11:17.165166 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d1015c-c37e-4972-9273-56deb6d14bab None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:11:17.176510 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application [None req-53d1015c-c37e-4972-9273-56deb6d14bab None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 12:11:17.176510 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:11:17.176510 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:11:17.176510 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:11:17.176510 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:11:17.176510 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:11:17.176510 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:11:17.176510 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:11:17.176510 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:11:17.176510 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:11:17.176510 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:11:17.176510 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:11:17.176510 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 15 12:11:17.176510 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 15 12:11:17.176510 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 15 12:11:17.177478 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 15 12:11:17.177478 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:11:17.177478 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:11:17.177478 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:11:17.177478 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:11:17.177478 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:11:17.177478 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:11:17.177478 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:11:17.177478 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application with Lock( Sep 15 12:11:17.177478 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:11:17.177478 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:11:17.177478 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:11:17.177478 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:11:17.177478 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:11:17.177478 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:11:17.177478 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:11:17.177478 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:11:17.178347 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 15 12:11:17.178347 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 15 12:11:17.178347 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 15 12:11:17.178347 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 15 12:11:17.178347 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 15 12:11:17.178347 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 15 12:11:17.178347 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 12:11:17.178347 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 15 12:11:17.178347 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 268/532] 10.222.0.15 () {56 vars in 1111 bytes} [Thu Sep 15 12:11:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:11:17.178818 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d1015c-c37e-4972-9273-56deb6d14bab None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:17.217063 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:11:17.222331 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d464f7-70fd-4f25-9e90-6b8a3a21b8af None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:17.262477 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad25c96a-af19-47f8-bba8-9160de8144e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:17.264118 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad25c96a-af19-47f8-bba8-9160de8144e7 None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:17.264429 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad25c96a-af19-47f8-bba8-9160de8144e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:17.264704 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad25c96a-af19-47f8-bba8-9160de8144e7 None admin] PATH_INFO: `/v3/projects` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:17.265960 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad25c96a-af19-47f8-bba8-9160de8144e7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81449) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:11:17.266099 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:11:17.266266 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad25c96a-af19-47f8-bba8-9160de8144e7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:11:17.269571 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad25c96a-af19-47f8-bba8-9160de8144e7 None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:11:17.292790 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 265/533] 10.222.0.15 () {56 vars in 1148 bytes} [Thu Sep 15 12:11:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:11:17.295114 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d1015c-c37e-4972-9273-56deb6d14bab None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:17.298431 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad25c96a-af19-47f8-bba8-9160de8144e7 None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:17.330474 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57e13fc-9382-4084-b88b-246172872794 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:17.331083 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57e13fc-9382-4084-b88b-246172872794 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:17.331362 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57e13fc-9382-4084-b88b-246172872794 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:17.331633 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57e13fc-9382-4084-b88b-246172872794 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:17.332355 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57e13fc-9382-4084-b88b-246172872794 None admin] RBAC: Adding query filter params (role.id=e1cecc29e028497f9cdaa04268c6a6ec, scope.project.id=540a32ef88764e8c8d9201ae22f6ddaf, user.id=c8191d3ccb874529b32f8d0026db629d) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:11:17.332663 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57e13fc-9382-4084-b88b-246172872794 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:11:17.334746 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57e13fc-9382-4084-b88b-246172872794 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:11:17.340054 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 269/534] 10.222.0.15 () {56 vars in 1374 bytes} [Thu Sep 15 12:11:17 2022] GET /identity/v3/role_assignments?user.id=c8191d3ccb874529b32f8d0026db629d&scope.project.id=540a32ef88764e8c8d9201ae22f6ddaf&role.id=e1cecc29e028497f9cdaa04268c6a6ec => generated 259 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:11:17.340856 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57e13fc-9382-4084-b88b-246172872794 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:17.420255 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad25c96a-af19-47f8-bba8-9160de8144e7 None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:17.470020 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c44de3-647b-444b-8c0b-128208ea963b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:17.471189 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c44de3-647b-444b-8c0b-128208ea963b None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:17.471485 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c44de3-647b-444b-8c0b-128208ea963b None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:17.471764 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c44de3-647b-444b-8c0b-128208ea963b None admin] PATH_INFO: `/v3/domains` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:17.472708 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c44de3-647b-444b-8c0b-128208ea963b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81449) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:11:17.473049 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c44de3-647b-444b-8c0b-128208ea963b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:11:17.474506 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c44de3-647b-444b-8c0b-128208ea963b None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:11:17.475115 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57e13fc-9382-4084-b88b-246172872794 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:17.492518 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 266/535] 10.222.0.15 () {56 vars in 1109 bytes} [Thu Sep 15 12:11:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:11:17.498775 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c44de3-647b-444b-8c0b-128208ea963b None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:17.524543 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9dd1ab9-968d-4b03-90b7-e4d5ec369a51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:17.525487 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9dd1ab9-968d-4b03-90b7-e4d5ec369a51 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:17.525956 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9dd1ab9-968d-4b03-90b7-e4d5ec369a51 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:17.526375 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9dd1ab9-968d-4b03-90b7-e4d5ec369a51 None admin] PATH_INFO: `/v3/projects` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:17.526697 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:11:17.527430 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9dd1ab9-968d-4b03-90b7-e4d5ec369a51 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:11:17.527949 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9dd1ab9-968d-4b03-90b7-e4d5ec369a51 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:11:17.530641 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9dd1ab9-968d-4b03-90b7-e4d5ec369a51 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:11:17.541070 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:11:17.548869 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 270/536] 10.222.0.15 () {56 vars in 1148 bytes} [Thu Sep 15 12:11:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:11:17.549965 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9dd1ab9-968d-4b03-90b7-e4d5ec369a51 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:17.619897 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c44de3-647b-444b-8c0b-128208ea963b None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:17.656050 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e738005e-f5e1-40e0-8f25-040a7b574269 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:17.657150 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e738005e-f5e1-40e0-8f25-040a7b574269 None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:17.657652 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e738005e-f5e1-40e0-8f25-040a7b574269 None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:17.658126 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e738005e-f5e1-40e0-8f25-040a7b574269 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:17.659044 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e738005e-f5e1-40e0-8f25-040a7b574269 None admin] RBAC: Adding query filter params (role.id=e1cecc29e028497f9cdaa04268c6a6ec, scope.project.id=540a32ef88764e8c8d9201ae22f6ddaf, user.id=eface89e261840a2abb4206be26455c7) {{(pid=81449) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:11:17.659581 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e738005e-f5e1-40e0-8f25-040a7b574269 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:11:17.661259 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e738005e-f5e1-40e0-8f25-040a7b574269 None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:11:17.667652 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 267/537] 10.222.0.15 () {56 vars in 1374 bytes} [Thu Sep 15 12:11:17 2022] GET /identity/v3/role_assignments?user.id=eface89e261840a2abb4206be26455c7&scope.project.id=540a32ef88764e8c8d9201ae22f6ddaf&role.id=e1cecc29e028497f9cdaa04268c6a6ec => generated 259 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:11:17.668709 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e738005e-f5e1-40e0-8f25-040a7b574269 None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:17.679169 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9dd1ab9-968d-4b03-90b7-e4d5ec369a51 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:17.736831 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067a755b-5a4d-4bbf-898c-54321e8739b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:17.736831 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067a755b-5a4d-4bbf-898c-54321e8739b8 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:17.736831 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067a755b-5a4d-4bbf-898c-54321e8739b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:17.736831 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067a755b-5a4d-4bbf-898c-54321e8739b8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:17.738610 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067a755b-5a4d-4bbf-898c-54321e8739b8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:11:17.740225 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067a755b-5a4d-4bbf-898c-54321e8739b8 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:11:17.755919 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application [None req-067a755b-5a4d-4bbf-898c-54321e8739b8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 12:11:17.755919 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:11:17.755919 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:11:17.755919 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:11:17.755919 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:11:17.755919 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:11:17.755919 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:11:17.755919 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:11:17.755919 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:11:17.755919 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:11:17.755919 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:11:17.755919 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:11:17.755919 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 15 12:11:17.755919 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 15 12:11:17.755919 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 15 12:11:17.757002 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 15 12:11:17.757002 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:11:17.757002 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:11:17.757002 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:11:17.757002 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:11:17.757002 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:11:17.757002 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:11:17.757002 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:11:17.757002 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application with Lock( Sep 15 12:11:17.757002 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:11:17.757002 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:11:17.757002 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:11:17.757002 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:11:17.757002 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:11:17.757002 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:11:17.757002 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:11:17.757002 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:11:17.757868 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 15 12:11:17.757868 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 15 12:11:17.757868 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 15 12:11:17.757868 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 15 12:11:17.757868 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 15 12:11:17.757868 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 15 12:11:17.757868 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 12:11:17.757868 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 15 12:11:17.762574 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 271/538] 10.222.0.15 () {56 vars in 1111 bytes} [Thu Sep 15 12:11:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:11:17.764808 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067a755b-5a4d-4bbf-898c-54321e8739b8 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:17.806566 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:11:17.840223 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:11:17.841093 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e738005e-f5e1-40e0-8f25-040a7b574269 None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:17.892799 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e09451-f80b-45c0-a6ae-d6ffdce5a679 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:17.893429 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e09451-f80b-45c0-a6ae-d6ffdce5a679 None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:17.893762 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e09451-f80b-45c0-a6ae-d6ffdce5a679 None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:17.894041 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e09451-f80b-45c0-a6ae-d6ffdce5a679 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:17.894764 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e09451-f80b-45c0-a6ae-d6ffdce5a679 None admin] RBAC: Adding query filter params (role.id=e1cecc29e028497f9cdaa04268c6a6ec, scope.project.id=540a32ef88764e8c8d9201ae22f6ddaf, user.id=bfbb4d6809064f9cb9e06626ed7d6814) {{(pid=81449) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:11:17.895073 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e09451-f80b-45c0-a6ae-d6ffdce5a679 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:11:17.896990 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067a755b-5a4d-4bbf-898c-54321e8739b8 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:17.897113 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e09451-f80b-45c0-a6ae-d6ffdce5a679 None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:11:17.903227 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 268/539] 10.222.0.15 () {56 vars in 1374 bytes} [Thu Sep 15 12:11:17 2022] GET /identity/v3/role_assignments?user.id=bfbb4d6809064f9cb9e06626ed7d6814&scope.project.id=540a32ef88764e8c8d9201ae22f6ddaf&role.id=e1cecc29e028497f9cdaa04268c6a6ec => generated 259 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:11:17.933528 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee7f01c-e212-4a2a-8a8c-48a66b24601e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:17.934160 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee7f01c-e212-4a2a-8a8c-48a66b24601e None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:17.934433 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee7f01c-e212-4a2a-8a8c-48a66b24601e None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:17.934699 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee7f01c-e212-4a2a-8a8c-48a66b24601e None admin] PATH_INFO: `/v3/projects` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:17.935415 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee7f01c-e212-4a2a-8a8c-48a66b24601e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:11:17.935718 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee7f01c-e212-4a2a-8a8c-48a66b24601e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:11:17.937227 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee7f01c-e212-4a2a-8a8c-48a66b24601e None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:11:17.951593 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 272/540] 10.222.0.15 () {56 vars in 1148 bytes} [Thu Sep 15 12:11:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:11:17.957523 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e09451-f80b-45c0-a6ae-d6ffdce5a679 None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:18.120086 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e09451-f80b-45c0-a6ae-d6ffdce5a679 None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:18.157802 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1645677-4517-48d8-8965-389d177ad697 None None] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:18.158087 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1645677-4517-48d8-8965-389d177ad697 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:18.158440 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1645677-4517-48d8-8965-389d177ad697 None None] PATH_INFO: `/` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:18.159944 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 273/541] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 12:11:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:11:18.164885 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338977bd-1150-47df-a4f2-bab448a5cff4 None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:18.165168 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338977bd-1150-47df-a4f2-bab448a5cff4 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:18.165438 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338977bd-1150-47df-a4f2-bab448a5cff4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:18.167420 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662766ae-fae6-40f4-ae26-0ae63d0a7f6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:18.168276 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662766ae-fae6-40f4-ae26-0ae63d0a7f6b None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:18.168954 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662766ae-fae6-40f4-ae26-0ae63d0a7f6b None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:18.169541 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662766ae-fae6-40f4-ae26-0ae63d0a7f6b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:18.170548 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-662766ae-fae6-40f4-ae26-0ae63d0a7f6b None admin] RBAC: Adding query filter params (role.id=e1cecc29e028497f9cdaa04268c6a6ec, scope.project.id=540a32ef88764e8c8d9201ae22f6ddaf, user.id=3aed8f78903f44cda9e5254aa46cf2b7) {{(pid=81449) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:11:18.171036 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-662766ae-fae6-40f4-ae26-0ae63d0a7f6b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:11:18.172929 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-662766ae-fae6-40f4-ae26-0ae63d0a7f6b None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:11:18.179633 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 269/542] 10.222.0.15 () {56 vars in 1374 bytes} [Thu Sep 15 12:11:17 2022] GET /identity/v3/role_assignments?user.id=3aed8f78903f44cda9e5254aa46cf2b7&scope.project.id=540a32ef88764e8c8d9201ae22f6ddaf&role.id=e1cecc29e028497f9cdaa04268c6a6ec => generated 259 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:11:18.210126 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-338977bd-1150-47df-a4f2-bab448a5cff4 None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:11:18.211269 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-338977bd-1150-47df-a4f2-bab448a5cff4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:11:18.318711 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 274/543] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 12:11:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 15 12:11:18.631022 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662766ae-fae6-40f4-ae26-0ae63d0a7f6b None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:18.644267 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:11:18.772177 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662766ae-fae6-40f4-ae26-0ae63d0a7f6b None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:18.809092 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40dc66e5-29c7-4ff5-a3be-f54c9b010bc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:18.809742 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40dc66e5-29c7-4ff5-a3be-f54c9b010bc0 None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:18.810009 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40dc66e5-29c7-4ff5-a3be-f54c9b010bc0 None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:18.810279 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40dc66e5-29c7-4ff5-a3be-f54c9b010bc0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:18.822699 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40dc66e5-29c7-4ff5-a3be-f54c9b010bc0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 12:11:18.828126 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application [None req-40dc66e5-29c7-4ff5-a3be-f54c9b010bc0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 15 12:11:18.828126 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:11:18.828126 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:11:18.828126 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:11:18.828126 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:11:18.828126 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:11:18.828126 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:11:18.828126 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:11:18.828126 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:11:18.828126 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:11:18.828126 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:11:18.828126 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:11:18.828126 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 12:11:18.828126 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 12:11:18.828126 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 12:11:18.828126 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 12:11:18.829035 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 12:11:18.829035 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 12:11:18.829035 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 12:11:18.829035 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 12:11:18.829035 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:11:18.829035 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:11:18.829035 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:11:18.829035 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:11:18.829035 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:11:18.829035 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:11:18.829035 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:11:18.829035 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application with Lock( Sep 15 12:11:18.829035 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:11:18.829035 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:11:18.829035 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:11:18.829035 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:11:18.830039 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:11:18.830039 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:11:18.830039 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:11:18.830039 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:11:18.830039 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 12:11:18.830039 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 12:11:18.830039 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 12:11:18.830039 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 12:11:18.830039 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 12:11:18.830039 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 12:11:18.830039 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 15 12:11:18.830039 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application Sep 15 12:11:18.830039 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 270/544] 10.222.0.15 () {56 vars in 1102 bytes} [Thu Sep 15 12:11:18 2022] GET /identity/v3/roles/service => generated 85 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:11:18.837733 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338977bd-1150-47df-a4f2-bab448a5cff4 None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:18.898886 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:11:18.963760 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338977bd-1150-47df-a4f2-bab448a5cff4 None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:19.013454 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3b52e6-9526-411c-8f99-d7bf47f0d1c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:19.014075 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3b52e6-9526-411c-8f99-d7bf47f0d1c8 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:19.014351 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3b52e6-9526-411c-8f99-d7bf47f0d1c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:19.014621 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3b52e6-9526-411c-8f99-d7bf47f0d1c8 None admin] PATH_INFO: `/v3/roles` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:19.015286 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b3b52e6-9526-411c-8f99-d7bf47f0d1c8 None admin] RBAC: Adding query filter params (name=service) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:11:19.015583 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b3b52e6-9526-411c-8f99-d7bf47f0d1c8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:11:19.016973 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b3b52e6-9526-411c-8f99-d7bf47f0d1c8 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:11:19.025453 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 275/545] 10.222.0.15 () {56 vars in 1103 bytes} [Thu Sep 15 12:11:18 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:11:19.031449 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40dc66e5-29c7-4ff5-a3be-f54c9b010bc0 None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:19.132736 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:11:19.173568 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40dc66e5-29c7-4ff5-a3be-f54c9b010bc0 None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:19.214386 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6138beb3-4adb-4684-a316-2e37fa4dcf48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:19.215012 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6138beb3-4adb-4684-a316-2e37fa4dcf48 None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:19.215288 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6138beb3-4adb-4684-a316-2e37fa4dcf48 None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:19.215555 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6138beb3-4adb-4684-a316-2e37fa4dcf48 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:19.226619 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6138beb3-4adb-4684-a316-2e37fa4dcf48 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:11:19.228085 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6138beb3-4adb-4684-a316-2e37fa4dcf48 None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:11:19.243460 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application [None req-6138beb3-4adb-4684-a316-2e37fa4dcf48 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 12:11:19.243460 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:11:19.243460 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:11:19.243460 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:11:19.243460 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:11:19.243460 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:11:19.243460 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:11:19.243460 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:11:19.243460 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:11:19.243460 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:11:19.243460 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:11:19.243460 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:11:19.243460 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 12:11:19.243460 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 12:11:19.243460 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 12:11:19.243460 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 12:11:19.244378 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:11:19.244378 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:11:19.244378 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:11:19.244378 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:11:19.244378 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:11:19.244378 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:11:19.244378 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:11:19.244378 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application with Lock( Sep 15 12:11:19.244378 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:11:19.244378 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:11:19.244378 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:11:19.244378 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:11:19.244378 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:11:19.244378 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:11:19.244378 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:11:19.244378 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:11:19.244378 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 12:11:19.245750 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 12:11:19.245750 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 12:11:19.245750 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application Sep 15 12:11:19.245750 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 271/546] 10.222.0.15 () {56 vars in 1108 bytes} [Thu Sep 15 12:11:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:11:19.251643 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3b52e6-9526-411c-8f99-d7bf47f0d1c8 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:19.402713 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3b52e6-9526-411c-8f99-d7bf47f0d1c8 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:19.416591 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:11:19.448054 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762b4d5b-e521-4a3e-a871-34c44468e6e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:19.448673 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762b4d5b-e521-4a3e-a871-34c44468e6e6 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:19.448978 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762b4d5b-e521-4a3e-a871-34c44468e6e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:19.449249 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762b4d5b-e521-4a3e-a871-34c44468e6e6 None admin] PATH_INFO: `/v3/domains` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:19.449891 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762b4d5b-e521-4a3e-a871-34c44468e6e6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:11:19.450193 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762b4d5b-e521-4a3e-a871-34c44468e6e6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:11:19.451560 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-762b4d5b-e521-4a3e-a871-34c44468e6e6 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:11:19.463831 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 276/547] 10.222.0.15 () {56 vars in 1109 bytes} [Thu Sep 15 12:11:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:11:19.469926 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6138beb3-4adb-4684-a316-2e37fa4dcf48 None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:19.603843 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6138beb3-4adb-4684-a316-2e37fa4dcf48 None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:19.627195 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:11:19.638779 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5faa76-ab92-4523-8bfd-6bd8bf33efdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:19.639609 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5faa76-ab92-4523-8bfd-6bd8bf33efdd None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:19.640072 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5faa76-ab92-4523-8bfd-6bd8bf33efdd None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:19.640403 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5faa76-ab92-4523-8bfd-6bd8bf33efdd None admin] PATH_INFO: `/v3/users/swift` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:19.653892 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5faa76-ab92-4523-8bfd-6bd8bf33efdd None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:11:19.655430 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5faa76-ab92-4523-8bfd-6bd8bf33efdd None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:11:19.667716 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application [None req-0e5faa76-ab92-4523-8bfd-6bd8bf33efdd None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 15 12:11:19.667716 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:11:19.667716 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:11:19.667716 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:11:19.667716 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:11:19.667716 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:11:19.667716 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:11:19.667716 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:11:19.667716 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:11:19.667716 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:11:19.667716 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:11:19.667716 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:11:19.667716 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 15 12:11:19.667716 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 15 12:11:19.667716 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 15 12:11:19.667716 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 15 12:11:19.670159 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:11:19.670159 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:11:19.670159 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 15 12:11:19.670159 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 12:11:19.670159 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 15 12:11:19.670159 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 12:11:19.670159 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:11:19.670159 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:11:19.670159 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:11:19.670159 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:11:19.670159 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:11:19.670159 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application with Lock( Sep 15 12:11:19.670159 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:11:19.670159 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:11:19.670159 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:11:19.670159 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:11:19.670159 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:11:19.670159 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:11:19.671491 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:11:19.671491 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:11:19.671491 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 15 12:11:19.671491 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 15 12:11:19.671491 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 15 12:11:19.671491 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 15 12:11:19.671491 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 15 12:11:19.671491 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 15 12:11:19.671491 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 15 12:11:19.671491 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application Sep 15 12:11:19.671491 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 272/548] 10.222.0.15 () {56 vars in 1096 bytes} [Thu Sep 15 12:11:19 2022] GET /identity/v3/users/swift => generated 83 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:11:19.678045 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762b4d5b-e521-4a3e-a871-34c44468e6e6 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:19.800825 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-762b4d5b-e521-4a3e-a871-34c44468e6e6 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:19.844197 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0bd121-23a6-4b66-bc1f-42345465decd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:19.845002 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0bd121-23a6-4b66-bc1f-42345465decd None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:19.845185 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0bd121-23a6-4b66-bc1f-42345465decd None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:19.845366 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0bd121-23a6-4b66-bc1f-42345465decd None admin] PATH_INFO: `/v3/users` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:19.846101 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0bd121-23a6-4b66-bc1f-42345465decd None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:11:19.846408 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0bd121-23a6-4b66-bc1f-42345465decd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:11:19.847848 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0bd121-23a6-4b66-bc1f-42345465decd None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:11:19.871326 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 277/549] 10.222.0.15 () {56 vars in 1135 bytes} [Thu Sep 15 12:11:19 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:11:19.877396 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5faa76-ab92-4523-8bfd-6bd8bf33efdd None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:20.026391 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5faa76-ab92-4523-8bfd-6bd8bf33efdd None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:20.068272 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58bcd34-bdab-44fb-9856-f65c010f13d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:20.068813 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58bcd34-bdab-44fb-9856-f65c010f13d0 None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:20.069147 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58bcd34-bdab-44fb-9856-f65c010f13d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:20.069421 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58bcd34-bdab-44fb-9856-f65c010f13d0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:20.143851 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:11:20.168873 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58bcd34-bdab-44fb-9856-f65c010f13d0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:11:20.170323 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58bcd34-bdab-44fb-9856-f65c010f13d0 None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:11:20.241496 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa908106-faf6-42ac-9e06-d6aeef698391 None None] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:20.242162 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa908106-faf6-42ac-9e06-d6aeef698391 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:20.242560 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa908106-faf6-42ac-9e06-d6aeef698391 None None] PATH_INFO: `/` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:20.248901 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 278/550] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 12:11:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:11:20.250429 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280b31dd-af41-4674-a18f-3abe5ea1c139 None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:20.250820 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280b31dd-af41-4674-a18f-3abe5ea1c139 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:20.251195 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280b31dd-af41-4674-a18f-3abe5ea1c139 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:20.292264 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-280b31dd-af41-4674-a18f-3abe5ea1c139 None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:11:20.294545 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-280b31dd-af41-4674-a18f-3abe5ea1c139 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:11:20.333038 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:11:20.339712 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 273/551] 10.222.0.15 () {58 vars in 1282 bytes} [Thu Sep 15 12:11:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 463 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 15 12:11:20.346101 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58bcd34-bdab-44fb-9856-f65c010f13d0 None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:20.403045 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 279/552] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 12:11:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 12:11:20.413064 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324ced7b-8102-4547-8785-90cf8bdfaf05 None None] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:20.413606 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324ced7b-8102-4547-8785-90cf8bdfaf05 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:20.414142 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324ced7b-8102-4547-8785-90cf8bdfaf05 None None] PATH_INFO: `/` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:20.415895 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 280/553] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 12:11:20 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:11:20.427804 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8d8d7d-3130-465d-a153-5ecfe9e64f31 None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:20.428408 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8d8d7d-3130-465d-a153-5ecfe9e64f31 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:20.428910 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8d8d7d-3130-465d-a153-5ecfe9e64f31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:20.452833 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:11:20.466624 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-3f8d8d7d-3130-465d-a153-5ecfe9e64f31 None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:11:20.467692 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58bcd34-bdab-44fb-9856-f65c010f13d0 None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:20.468396 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-3f8d8d7d-3130-465d-a153-5ecfe9e64f31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:11:20.509077 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ab7fd3-6a85-443e-981a-b027507c104a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:20.509793 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ab7fd3-6a85-443e-981a-b027507c104a None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:20.510074 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ab7fd3-6a85-443e-981a-b027507c104a None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:20.510345 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ab7fd3-6a85-443e-981a-b027507c104a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:20.521876 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ab7fd3-6a85-443e-981a-b027507c104a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:11:20.523437 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ab7fd3-6a85-443e-981a-b027507c104a None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:11:20.539103 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application [None req-d7ab7fd3-6a85-443e-981a-b027507c104a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 12:11:20.539103 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:11:20.539103 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:11:20.539103 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:11:20.539103 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:11:20.539103 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:11:20.539103 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:11:20.539103 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:11:20.539103 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:11:20.539103 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:11:20.539103 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:11:20.539103 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:11:20.539103 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 12:11:20.539103 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 12:11:20.539103 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 12:11:20.539103 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 12:11:20.540068 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:11:20.540068 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:11:20.540068 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:11:20.540068 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:11:20.540068 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:11:20.540068 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:11:20.540068 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:11:20.540068 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application with Lock( Sep 15 12:11:20.540068 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:11:20.540068 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:11:20.540068 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:11:20.540068 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:11:20.540068 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:11:20.540068 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:11:20.540068 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:11:20.540068 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:11:20.540068 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 12:11:20.540992 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 12:11:20.540992 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 12:11:20.540992 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application Sep 15 12:11:20.544816 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 274/554] 10.222.0.15 () {56 vars in 1108 bytes} [Thu Sep 15 12:11:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:11:20.546076 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ab7fd3-6a85-443e-981a-b027507c104a None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:20.654611 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 281/555] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 12:11:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 15 12:11:20.664287 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ab7fd3-6a85-443e-981a-b027507c104a None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:20.718623 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f962d41-91e2-42d6-aa58-b3dff8d1b59d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:20.719245 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f962d41-91e2-42d6-aa58-b3dff8d1b59d None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:20.719527 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f962d41-91e2-42d6-aa58-b3dff8d1b59d None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:20.719804 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f962d41-91e2-42d6-aa58-b3dff8d1b59d None admin] PATH_INFO: `/v3/domains` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:20.720449 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f962d41-91e2-42d6-aa58-b3dff8d1b59d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81449) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:11:20.720769 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f962d41-91e2-42d6-aa58-b3dff8d1b59d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:11:20.722144 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f962d41-91e2-42d6-aa58-b3dff8d1b59d None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:11:20.723597 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:11:20.741842 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 275/556] 10.222.0.15 () {56 vars in 1109 bytes} [Thu Sep 15 12:11:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 12:11:20.749839 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f962d41-91e2-42d6-aa58-b3dff8d1b59d None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:20.749978 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8d8d7d-3130-465d-a153-5ecfe9e64f31 None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:20.874980 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:11:20.889245 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8d8d7d-3130-465d-a153-5ecfe9e64f31 None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:20.893990 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f962d41-91e2-42d6-aa58-b3dff8d1b59d None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:20.931059 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc5f48c-ba21-484a-9ae3-aed076dcf278 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:20.931799 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc5f48c-ba21-484a-9ae3-aed076dcf278 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:20.932080 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc5f48c-ba21-484a-9ae3-aed076dcf278 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:20.932351 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc5f48c-ba21-484a-9ae3-aed076dcf278 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:20.935233 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac43854b-2f06-43f2-8a7e-378472f870d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:20.936217 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac43854b-2f06-43f2-8a7e-378472f870d1 None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:20.936709 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac43854b-2f06-43f2-8a7e-378472f870d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:20.937220 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac43854b-2f06-43f2-8a7e-378472f870d1 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:20.940262 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc5f48c-ba21-484a-9ae3-aed076dcf278 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:11:20.942019 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc5f48c-ba21-484a-9ae3-aed076dcf278 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:11:20.948860 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac43854b-2f06-43f2-8a7e-378472f870d1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 12:11:20.953757 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application [None req-edc5f48c-ba21-484a-9ae3-aed076dcf278 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 12:11:20.953757 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:11:20.953757 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:11:20.953757 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:11:20.953757 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:11:20.953757 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:11:20.953757 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:11:20.953757 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:11:20.953757 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:11:20.953757 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:11:20.953757 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:11:20.953757 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:11:20.953757 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 15 12:11:20.953757 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 15 12:11:20.953757 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 15 12:11:20.954626 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 15 12:11:20.954626 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:11:20.954626 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:11:20.954626 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:11:20.954626 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:11:20.954626 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:11:20.954626 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:11:20.954626 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:11:20.954626 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application with Lock( Sep 15 12:11:20.954626 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:11:20.954626 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:11:20.954626 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:11:20.954626 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:11:20.954626 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:11:20.954626 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:11:20.954626 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:11:20.954626 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:11:20.955520 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 15 12:11:20.955520 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 15 12:11:20.955520 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 15 12:11:20.955520 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 15 12:11:20.955520 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 15 12:11:20.955520 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 15 12:11:20.955520 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 12:11:20.955520 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 15 12:11:20.955520 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 282/557] 10.222.0.15 () {56 vars in 1111 bytes} [Thu Sep 15 12:11:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:11:20.956443 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18f6e1a-b819-4886-bf97-9a2dd88f67e4 None None] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:20.956538 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18f6e1a-b819-4886-bf97-9a2dd88f67e4 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:20.956778 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application [None req-ac43854b-2f06-43f2-8a7e-378472f870d1 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 15 12:11:20.956778 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:11:20.956778 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:11:20.956778 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:11:20.956778 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:11:20.956778 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:11:20.956778 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:11:20.956778 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:11:20.956778 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:11:20.956778 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:11:20.956778 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:11:20.956778 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:11:20.956778 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 12:11:20.956778 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 12:11:20.956778 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 12:11:20.956778 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 12:11:20.958175 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 12:11:20.958175 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 12:11:20.958175 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 12:11:20.958175 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 12:11:20.958175 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:11:20.958175 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:11:20.958175 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:11:20.958175 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:11:20.958175 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:11:20.958175 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:11:20.958175 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:11:20.958175 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application with Lock( Sep 15 12:11:20.958175 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:11:20.958175 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:11:20.958175 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:11:20.958175 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:11:20.959049 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:11:20.959049 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:11:20.959049 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:11:20.959049 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:11:20.959049 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 12:11:20.959049 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 12:11:20.959049 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 12:11:20.959049 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 12:11:20.959049 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 12:11:20.959049 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 12:11:20.959049 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 15 12:11:20.959049 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application Sep 15 12:11:20.965336 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18f6e1a-b819-4886-bf97-9a2dd88f67e4 None None] PATH_INFO: `/` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:20.965500 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 276/558] 10.222.0.15 () {56 vars in 1102 bytes} [Thu Sep 15 12:11:20 2022] GET /identity/v3/roles/service => generated 85 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:11:20.965500 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2261b02-7efc-47e4-910c-d6a9db00d8b1 None None] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:20.965500 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2261b02-7efc-47e4-910c-d6a9db00d8b1 None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:20.965500 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2261b02-7efc-47e4-910c-d6a9db00d8b1 None None] PATH_INFO: `/` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:20.966034 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 283/559] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 12:11:20 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:11:20.966034 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18f6e1a-b819-4886-bf97-9a2dd88f67e4 None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:20.966300 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 277/560] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 12:11:20 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:11:20.966300 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0382dd-f588-46ad-b1e5-3b29e1c3abff None None] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:20.966300 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0382dd-f588-46ad-b1e5-3b29e1c3abff None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:20.966703 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad0382dd-f588-46ad-b1e5-3b29e1c3abff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:20.991750 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.auth.core [None req-ad0382dd-f588-46ad-b1e5-3b29e1c3abff None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81449) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:11:20.992994 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-ad0382dd-f588-46ad-b1e5-3b29e1c3abff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:11:21.047612 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:11:21.066106 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 278/561] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 12:11:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 12:11:21.066925 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0382dd-f588-46ad-b1e5-3b29e1c3abff None None] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:21.083784 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18f6e1a-b819-4886-bf97-9a2dd88f67e4 None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:21.120872 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6967db-7dfc-49e5-b7f4-399908acb037 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:21.121479 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6967db-7dfc-49e5-b7f4-399908acb037 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:21.121878 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6967db-7dfc-49e5-b7f4-399908acb037 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:21.122153 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6967db-7dfc-49e5-b7f4-399908acb037 None admin] PATH_INFO: `/v3/projects` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:21.122828 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6967db-7dfc-49e5-b7f4-399908acb037 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:11:21.123135 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6967db-7dfc-49e5-b7f4-399908acb037 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:11:21.124678 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6967db-7dfc-49e5-b7f4-399908acb037 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:11:21.141942 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 284/562] 10.222.0.15 () {56 vars in 1148 bytes} [Thu Sep 15 12:11:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:11:21.141942 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2bbdd0-a42b-486f-b06a-a4d809797983 None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:21.141942 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2bbdd0-a42b-486f-b06a-a4d809797983 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:21.141942 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2bbdd0-a42b-486f-b06a-a4d809797983 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:21.157446 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-db2bbdd0-a42b-486f-b06a-a4d809797983 None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:11:21.158621 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-db2bbdd0-a42b-486f-b06a-a4d809797983 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:11:21.160810 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:11:21.169679 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad0382dd-f588-46ad-b1e5-3b29e1c3abff None None] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:21.203083 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0262d23-1c72-4272-996d-13833ec4222c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:21.203619 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0262d23-1c72-4272-996d-13833ec4222c None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:21.203901 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0262d23-1c72-4272-996d-13833ec4222c None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:21.204175 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0262d23-1c72-4272-996d-13833ec4222c None admin] PATH_INFO: `/v3/roles` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:21.204908 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0262d23-1c72-4272-996d-13833ec4222c None admin] RBAC: Adding query filter params (name=service) {{(pid=81449) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:11:21.205211 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0262d23-1c72-4272-996d-13833ec4222c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:11:21.206576 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0262d23-1c72-4272-996d-13833ec4222c None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:11:21.214982 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 279/563] 10.222.0.15 () {56 vars in 1103 bytes} [Thu Sep 15 12:11:21 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:11:21.215763 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0262d23-1c72-4272-996d-13833ec4222c None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:21.237832 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 285/564] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 12:11:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 12:11:21.239022 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2bbdd0-a42b-486f-b06a-a4d809797983 None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:21.293275 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:11:21.352583 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2bbdd0-a42b-486f-b06a-a4d809797983 None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:21.365752 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0262d23-1c72-4272-996d-13833ec4222c None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:21.388716 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9059efd9-724c-4661-9653-8674e248dcdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:21.389353 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9059efd9-724c-4661-9653-8674e248dcdf None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:21.389636 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9059efd9-724c-4661-9653-8674e248dcdf None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:21.389905 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9059efd9-724c-4661-9653-8674e248dcdf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:21.390627 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9059efd9-724c-4661-9653-8674e248dcdf None admin] RBAC: Adding query filter params (role.id=e1cecc29e028497f9cdaa04268c6a6ec, scope.project.id=540a32ef88764e8c8d9201ae22f6ddaf, user.id=efa0d3dc5b7540d59fdd59bbf64a04b6) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:11:21.390934 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9059efd9-724c-4661-9653-8674e248dcdf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:11:21.392434 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9059efd9-724c-4661-9653-8674e248dcdf None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:11:21.398732 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 286/565] 10.222.0.15 () {56 vars in 1374 bytes} [Thu Sep 15 12:11:21 2022] GET /identity/v3/role_assignments?user.id=efa0d3dc5b7540d59fdd59bbf64a04b6&scope.project.id=540a32ef88764e8c8d9201ae22f6ddaf&role.id=e1cecc29e028497f9cdaa04268c6a6ec => generated 259 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:11:21.399516 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9059efd9-724c-4661-9653-8674e248dcdf None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:21.403699 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e241e1ac-c283-4f5e-acec-387f969ecba9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:21.404719 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e241e1ac-c283-4f5e-acec-387f969ecba9 None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:21.405251 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e241e1ac-c283-4f5e-acec-387f969ecba9 None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:21.405729 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e241e1ac-c283-4f5e-acec-387f969ecba9 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:21.408651 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:11:21.421914 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e241e1ac-c283-4f5e-acec-387f969ecba9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 12:11:21.429285 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application [None req-e241e1ac-c283-4f5e-acec-387f969ecba9 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 15 12:11:21.429285 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:11:21.429285 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:11:21.429285 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:11:21.429285 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:11:21.429285 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:11:21.429285 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:11:21.429285 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:11:21.429285 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:11:21.429285 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:11:21.429285 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:11:21.429285 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:11:21.429285 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 12:11:21.429285 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 12:11:21.429285 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 12:11:21.429285 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 12:11:21.430283 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 12:11:21.430283 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 12:11:21.430283 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 12:11:21.430283 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 12:11:21.430283 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:11:21.430283 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:11:21.430283 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:11:21.430283 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:11:21.430283 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:11:21.430283 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:11:21.430283 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:11:21.430283 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application with Lock( Sep 15 12:11:21.430283 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:11:21.430283 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:11:21.430283 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:11:21.430283 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:11:21.431118 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:11:21.431118 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:11:21.431118 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:11:21.431118 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:11:21.431118 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 12:11:21.431118 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 12:11:21.431118 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 12:11:21.431118 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 12:11:21.431118 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 12:11:21.431118 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 12:11:21.431118 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 15 12:11:21.431118 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application Sep 15 12:11:21.431118 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 280/566] 10.222.0.15 () {56 vars in 1102 bytes} [Thu Sep 15 12:11:21 2022] GET /identity/v3/roles/service => generated 85 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:11:21.431853 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e241e1ac-c283-4f5e-acec-387f969ecba9 None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:21.531151 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9059efd9-724c-4661-9653-8674e248dcdf None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:21.565521 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e241e1ac-c283-4f5e-acec-387f969ecba9 None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:21.568366 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515680e3-7469-45a7-9876-28d042eeb70c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:21.569144 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515680e3-7469-45a7-9876-28d042eeb70c None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:21.569483 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515680e3-7469-45a7-9876-28d042eeb70c None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:21.569760 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515680e3-7469-45a7-9876-28d042eeb70c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:21.574672 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:11:21.584019 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515680e3-7469-45a7-9876-28d042eeb70c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:11:21.585553 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515680e3-7469-45a7-9876-28d042eeb70c None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:11:21.603410 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application [None req-515680e3-7469-45a7-9876-28d042eeb70c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 12:11:21.603410 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:11:21.603410 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:11:21.603410 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:11:21.603410 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:11:21.603410 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:11:21.603410 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:11:21.603410 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:11:21.603410 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:11:21.603410 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:11:21.603410 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:11:21.603410 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:11:21.603410 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 12:11:21.603410 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 12:11:21.603410 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 12:11:21.603410 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 12:11:21.605398 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:11:21.605398 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:11:21.605398 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:11:21.605398 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:11:21.605398 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:11:21.605398 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:11:21.605398 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:11:21.605398 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application with Lock( Sep 15 12:11:21.605398 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:11:21.605398 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:11:21.605398 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:11:21.605398 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:11:21.605398 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:11:21.605398 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:11:21.605398 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:11:21.605398 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:11:21.605398 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 12:11:21.606327 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 12:11:21.606327 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 12:11:21.606327 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 15 12:11:21.606327 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 287/567] 10.222.0.15 () {56 vars in 1108 bytes} [Thu Sep 15 12:11:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:11:21.606327 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515680e3-7469-45a7-9876-28d042eeb70c None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:21.612306 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc83d4b-ed22-4fbd-99fb-c7534094970b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:21.612985 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc83d4b-ed22-4fbd-99fb-c7534094970b None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:21.613268 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc83d4b-ed22-4fbd-99fb-c7534094970b None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:21.613538 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc83d4b-ed22-4fbd-99fb-c7534094970b None admin] PATH_INFO: `/v3/roles/service` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:21.624169 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc83d4b-ed22-4fbd-99fb-c7534094970b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 12:11:21.629416 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application [None req-efc83d4b-ed22-4fbd-99fb-c7534094970b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 15 12:11:21.629416 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:11:21.629416 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:11:21.629416 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:11:21.629416 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:11:21.629416 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:11:21.629416 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:11:21.629416 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:11:21.629416 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:11:21.629416 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:11:21.629416 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:11:21.629416 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:11:21.629416 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 12:11:21.629416 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 12:11:21.629416 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 12:11:21.629416 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 12:11:21.630869 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 12:11:21.630869 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 12:11:21.630869 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 12:11:21.630869 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 12:11:21.630869 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:11:21.630869 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:11:21.630869 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:11:21.630869 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:11:21.630869 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:11:21.630869 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:11:21.630869 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:11:21.630869 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application with Lock( Sep 15 12:11:21.630869 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:11:21.630869 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:11:21.630869 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:11:21.630869 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:11:21.631806 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:11:21.631806 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:11:21.631806 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:11:21.631806 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:11:21.631806 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 12:11:21.631806 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 12:11:21.631806 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 12:11:21.631806 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 12:11:21.631806 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 12:11:21.631806 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 12:11:21.631806 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 15 12:11:21.631806 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application Sep 15 12:11:21.631806 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 281/568] 10.222.0.15 () {56 vars in 1102 bytes} [Thu Sep 15 12:11:21 2022] GET /identity/v3/roles/service => generated 85 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:11:21.632510 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc83d4b-ed22-4fbd-99fb-c7534094970b None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:21.672541 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:11:21.720143 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515680e3-7469-45a7-9876-28d042eeb70c None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:21.739817 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc83d4b-ed22-4fbd-99fb-c7534094970b None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:21.758808 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8949682-3509-4a36-8b0c-5867932cf2c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:21.759730 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8949682-3509-4a36-8b0c-5867932cf2c8 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:21.760230 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8949682-3509-4a36-8b0c-5867932cf2c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:21.760694 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8949682-3509-4a36-8b0c-5867932cf2c8 None admin] PATH_INFO: `/v3/roles` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:21.761390 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8949682-3509-4a36-8b0c-5867932cf2c8 None admin] RBAC: Adding query filter params (name=service) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:11:21.761693 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8949682-3509-4a36-8b0c-5867932cf2c8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:11:21.763045 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8949682-3509-4a36-8b0c-5867932cf2c8 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:11:21.771566 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 288/569] 10.222.0.15 () {56 vars in 1103 bytes} [Thu Sep 15 12:11:21 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:11:21.772369 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8949682-3509-4a36-8b0c-5867932cf2c8 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:21.781220 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b07680-60ac-429e-820d-339c93a0c4e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:21.781806 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b07680-60ac-429e-820d-339c93a0c4e6 None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:21.782087 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b07680-60ac-429e-820d-339c93a0c4e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:21.782362 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b07680-60ac-429e-820d-339c93a0c4e6 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:21.793224 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b07680-60ac-429e-820d-339c93a0c4e6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 12:11:21.798410 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application [None req-46b07680-60ac-429e-820d-339c93a0c4e6 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 15 12:11:21.798410 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:11:21.798410 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:11:21.798410 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:11:21.798410 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:11:21.798410 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:11:21.798410 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:11:21.798410 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:11:21.798410 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:11:21.798410 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:11:21.798410 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:11:21.798410 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:11:21.798410 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 12:11:21.798410 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 12:11:21.798410 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 12:11:21.798410 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 12:11:21.799321 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 12:11:21.799321 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 12:11:21.799321 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 12:11:21.799321 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 12:11:21.799321 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:11:21.799321 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:11:21.799321 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:11:21.799321 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:11:21.799321 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:11:21.799321 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:11:21.799321 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:11:21.799321 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application with Lock( Sep 15 12:11:21.799321 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:11:21.799321 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:11:21.799321 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:11:21.799321 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:11:21.800201 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:11:21.800201 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:11:21.800201 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:11:21.800201 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:11:21.800201 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 12:11:21.800201 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 12:11:21.800201 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 12:11:21.800201 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 12:11:21.800201 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 12:11:21.800201 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 12:11:21.800201 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 15 12:11:21.800201 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application Sep 15 12:11:21.800201 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 282/570] 10.222.0.15 () {56 vars in 1102 bytes} [Thu Sep 15 12:11:21 2022] GET /identity/v3/roles/service => generated 85 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:11:21.800938 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b07680-60ac-429e-820d-339c93a0c4e6 None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:21.832420 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:11:21.886026 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8949682-3509-4a36-8b0c-5867932cf2c8 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:21.903164 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:11:21.911970 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b07680-60ac-429e-820d-339c93a0c4e6 None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:21.922835 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f53260-6240-4bc0-8784-1d1d9ff1bbf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:21.923458 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f53260-6240-4bc0-8784-1d1d9ff1bbf7 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:21.923738 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f53260-6240-4bc0-8784-1d1d9ff1bbf7 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:21.924012 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f53260-6240-4bc0-8784-1d1d9ff1bbf7 None admin] PATH_INFO: `/v3/domains` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:21.924708 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f53260-6240-4bc0-8784-1d1d9ff1bbf7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:11:21.925231 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f53260-6240-4bc0-8784-1d1d9ff1bbf7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:11:21.927554 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f53260-6240-4bc0-8784-1d1d9ff1bbf7 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:11:21.939457 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 289/571] 10.222.0.15 () {56 vars in 1109 bytes} [Thu Sep 15 12:11:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:11:21.940278 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f53260-6240-4bc0-8784-1d1d9ff1bbf7 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:21.950088 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4aab1c4-7b2a-49ed-a55a-fed328dde6e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:21.950905 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4aab1c4-7b2a-49ed-a55a-fed328dde6e8 None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:21.951181 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4aab1c4-7b2a-49ed-a55a-fed328dde6e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:21.951454 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4aab1c4-7b2a-49ed-a55a-fed328dde6e8 None admin] PATH_INFO: `/v3/roles` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:21.952108 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4aab1c4-7b2a-49ed-a55a-fed328dde6e8 None admin] RBAC: Adding query filter params (name=service) {{(pid=81449) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:11:21.952407 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4aab1c4-7b2a-49ed-a55a-fed328dde6e8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:11:21.953809 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4aab1c4-7b2a-49ed-a55a-fed328dde6e8 None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:11:21.962273 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 283/572] 10.222.0.15 () {56 vars in 1103 bytes} [Thu Sep 15 12:11:21 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:11:21.963084 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4aab1c4-7b2a-49ed-a55a-fed328dde6e8 None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:22.056011 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:11:22.056586 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f53260-6240-4bc0-8784-1d1d9ff1bbf7 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:22.068864 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4aab1c4-7b2a-49ed-a55a-fed328dde6e8 None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:22.098894 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8460008-468a-4469-9b15-4d89a9d21867 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:22.099537 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8460008-468a-4469-9b15-4d89a9d21867 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:22.099861 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8460008-468a-4469-9b15-4d89a9d21867 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:22.100134 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8460008-468a-4469-9b15-4d89a9d21867 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:22.102753 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1536cb95-3e0e-45ae-8a35-3783b671e89b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:22.103451 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1536cb95-3e0e-45ae-8a35-3783b671e89b None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:22.103734 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1536cb95-3e0e-45ae-8a35-3783b671e89b None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:22.104006 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1536cb95-3e0e-45ae-8a35-3783b671e89b None admin] PATH_INFO: `/v3/roles` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:22.104952 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1536cb95-3e0e-45ae-8a35-3783b671e89b None admin] RBAC: Adding query filter params (name=service) {{(pid=81449) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:11:22.105258 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1536cb95-3e0e-45ae-8a35-3783b671e89b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:11:22.106617 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1536cb95-3e0e-45ae-8a35-3783b671e89b None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:11:22.112497 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8460008-468a-4469-9b15-4d89a9d21867 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:11:22.113846 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8460008-468a-4469-9b15-4d89a9d21867 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:11:22.116569 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 284/573] 10.222.0.15 () {56 vars in 1103 bytes} [Thu Sep 15 12:11:21 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:11:22.117339 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1536cb95-3e0e-45ae-8a35-3783b671e89b None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:22.135392 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application [None req-a8460008-468a-4469-9b15-4d89a9d21867 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 12:11:22.135392 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:11:22.135392 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:11:22.135392 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:11:22.135392 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:11:22.135392 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:11:22.135392 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:11:22.135392 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:11:22.135392 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:11:22.135392 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:11:22.135392 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:11:22.135392 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:11:22.135392 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 12:11:22.135392 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 12:11:22.135392 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 12:11:22.135392 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 12:11:22.136433 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:11:22.136433 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:11:22.136433 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:11:22.136433 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:11:22.136433 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:11:22.136433 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:11:22.136433 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:11:22.136433 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application with Lock( Sep 15 12:11:22.136433 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:11:22.136433 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:11:22.136433 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:11:22.136433 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:11:22.136433 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:11:22.136433 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:11:22.136433 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:11:22.136433 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:11:22.136433 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 12:11:22.137340 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 12:11:22.137340 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 12:11:22.137340 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 15 12:11:22.137340 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 290/574] 10.222.0.15 () {56 vars in 1108 bytes} [Thu Sep 15 12:11:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:11:22.137340 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8460008-468a-4469-9b15-4d89a9d21867 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:22.179278 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:11:22.222642 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1536cb95-3e0e-45ae-8a35-3783b671e89b None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:22.255830 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8460008-468a-4469-9b15-4d89a9d21867 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:22.257540 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973803ad-e6d3-404f-bc5a-0cc8ce92ee58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:22.258174 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973803ad-e6d3-404f-bc5a-0cc8ce92ee58 None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:22.258446 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973803ad-e6d3-404f-bc5a-0cc8ce92ee58 None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:22.258712 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973803ad-e6d3-404f-bc5a-0cc8ce92ee58 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:22.266304 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973803ad-e6d3-404f-bc5a-0cc8ce92ee58 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:11:22.267742 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973803ad-e6d3-404f-bc5a-0cc8ce92ee58 None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:11:22.279979 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application [None req-973803ad-e6d3-404f-bc5a-0cc8ce92ee58 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 15 12:11:22.279979 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:11:22.279979 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:11:22.279979 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:11:22.279979 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:11:22.279979 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:11:22.279979 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:11:22.279979 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:11:22.279979 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:11:22.279979 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:11:22.279979 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:11:22.279979 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:11:22.279979 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 15 12:11:22.279979 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 15 12:11:22.279979 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 15 12:11:22.279979 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 15 12:11:22.281093 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:11:22.281093 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:11:22.281093 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 15 12:11:22.281093 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 12:11:22.281093 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 15 12:11:22.281093 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 12:11:22.281093 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:11:22.281093 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:11:22.281093 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:11:22.281093 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:11:22.281093 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:11:22.281093 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application with Lock( Sep 15 12:11:22.281093 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:11:22.281093 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:11:22.281093 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:11:22.281093 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:11:22.281093 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:11:22.281093 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:11:22.282028 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:11:22.282028 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:11:22.282028 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 15 12:11:22.282028 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 15 12:11:22.282028 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 15 12:11:22.282028 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 15 12:11:22.282028 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 15 12:11:22.282028 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 15 12:11:22.282028 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 15 12:11:22.282028 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application Sep 15 12:11:22.282028 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 285/575] 10.222.0.15 () {56 vars in 1093 bytes} [Thu Sep 15 12:11:22 2022] GET /identity/v3/users/nova => generated 82 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:11:22.282028 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973803ad-e6d3-404f-bc5a-0cc8ce92ee58 None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:22.299274 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:11:22.301498 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f48f13f-e1fb-466a-8031-6a7c8a87cf01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:22.303049 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f48f13f-e1fb-466a-8031-6a7c8a87cf01 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:22.303781 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f48f13f-e1fb-466a-8031-6a7c8a87cf01 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:22.304016 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f48f13f-e1fb-466a-8031-6a7c8a87cf01 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:22.316449 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f48f13f-e1fb-466a-8031-6a7c8a87cf01 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:11:22.317880 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f48f13f-e1fb-466a-8031-6a7c8a87cf01 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:11:22.331765 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application [None req-5f48f13f-e1fb-466a-8031-6a7c8a87cf01 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 12:11:22.331765 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:11:22.331765 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:11:22.331765 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:11:22.331765 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:11:22.331765 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:11:22.331765 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:11:22.331765 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:11:22.331765 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:11:22.331765 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:11:22.331765 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:11:22.331765 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:11:22.331765 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 12:11:22.331765 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 12:11:22.331765 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 12:11:22.331765 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 12:11:22.332922 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:11:22.332922 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:11:22.332922 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:11:22.332922 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:11:22.332922 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:11:22.332922 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:11:22.332922 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:11:22.332922 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application with Lock( Sep 15 12:11:22.332922 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:11:22.332922 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:11:22.332922 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:11:22.332922 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:11:22.332922 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:11:22.332922 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:11:22.332922 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:11:22.332922 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:11:22.332922 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 12:11:22.333996 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 12:11:22.333996 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 12:11:22.333996 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 15 12:11:22.333996 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 291/576] 10.222.0.15 () {56 vars in 1108 bytes} [Thu Sep 15 12:11:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:11:22.333996 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f48f13f-e1fb-466a-8031-6a7c8a87cf01 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:22.393645 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973803ad-e6d3-404f-bc5a-0cc8ce92ee58 None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:22.422022 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:11:22.429291 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f66986-3efe-43bf-a32e-c7b58daf3f87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:22.429920 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f66986-3efe-43bf-a32e-c7b58daf3f87 None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:22.430196 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f66986-3efe-43bf-a32e-c7b58daf3f87 None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:22.430882 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f66986-3efe-43bf-a32e-c7b58daf3f87 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:22.442227 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f48f13f-e1fb-466a-8031-6a7c8a87cf01 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:22.442378 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f66986-3efe-43bf-a32e-c7b58daf3f87 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:11:22.443693 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f66986-3efe-43bf-a32e-c7b58daf3f87 None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:11:22.457465 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application [None req-c7f66986-3efe-43bf-a32e-c7b58daf3f87 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 12:11:22.457465 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:11:22.457465 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:11:22.457465 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:11:22.457465 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:11:22.457465 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:11:22.457465 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:11:22.457465 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:11:22.457465 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:11:22.457465 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:11:22.457465 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:11:22.457465 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:11:22.457465 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 12:11:22.457465 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 12:11:22.457465 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 12:11:22.457465 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 12:11:22.458481 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:11:22.458481 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:11:22.458481 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:11:22.458481 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:11:22.458481 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:11:22.458481 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:11:22.458481 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:11:22.458481 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application with Lock( Sep 15 12:11:22.458481 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:11:22.458481 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:11:22.458481 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:11:22.458481 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:11:22.458481 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:11:22.458481 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:11:22.458481 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:11:22.458481 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:11:22.458481 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 12:11:22.459368 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 12:11:22.459368 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 12:11:22.459368 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application Sep 15 12:11:22.459368 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 286/577] 10.222.0.15 () {56 vars in 1108 bytes} [Thu Sep 15 12:11:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:11:22.460237 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f66986-3efe-43bf-a32e-c7b58daf3f87 None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:22.482263 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce86903-33f0-41f4-8c36-55854eaaddd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:22.482851 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce86903-33f0-41f4-8c36-55854eaaddd8 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:22.483126 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce86903-33f0-41f4-8c36-55854eaaddd8 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:22.483395 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce86903-33f0-41f4-8c36-55854eaaddd8 None admin] PATH_INFO: `/v3/domains` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:22.484026 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce86903-33f0-41f4-8c36-55854eaaddd8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:11:22.484323 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce86903-33f0-41f4-8c36-55854eaaddd8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:11:22.485928 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce86903-33f0-41f4-8c36-55854eaaddd8 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:11:22.499635 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 292/578] 10.222.0.15 () {56 vars in 1109 bytes} [Thu Sep 15 12:11:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:11:22.504717 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce86903-33f0-41f4-8c36-55854eaaddd8 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:22.522506 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:11:22.564404 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f66986-3efe-43bf-a32e-c7b58daf3f87 None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:22.598453 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8232be54-41f7-4e7a-b6d3-dcfed1d21679 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:22.599520 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8232be54-41f7-4e7a-b6d3-dcfed1d21679 None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:22.599988 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8232be54-41f7-4e7a-b6d3-dcfed1d21679 None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:22.600095 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8232be54-41f7-4e7a-b6d3-dcfed1d21679 None admin] PATH_INFO: `/v3/users` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:22.600823 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8232be54-41f7-4e7a-b6d3-dcfed1d21679 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81449) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:11:22.601126 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8232be54-41f7-4e7a-b6d3-dcfed1d21679 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:11:22.602539 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8232be54-41f7-4e7a-b6d3-dcfed1d21679 None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:11:22.611595 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce86903-33f0-41f4-8c36-55854eaaddd8 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:22.627748 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 287/579] 10.222.0.15 () {56 vars in 1133 bytes} [Thu Sep 15 12:11:22 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:11:22.627748 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8232be54-41f7-4e7a-b6d3-dcfed1d21679 None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:22.653893 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076cc513-da64-4db6-b4b4-b078f22e9bea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:22.654533 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076cc513-da64-4db6-b4b4-b078f22e9bea None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:22.654813 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076cc513-da64-4db6-b4b4-b078f22e9bea None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:22.655082 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076cc513-da64-4db6-b4b4-b078f22e9bea None admin] PATH_INFO: `/v3/domains` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:22.655721 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-076cc513-da64-4db6-b4b4-b078f22e9bea None admin] RBAC: Adding query filter params (name=Default) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:11:22.656019 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-076cc513-da64-4db6-b4b4-b078f22e9bea None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:11:22.657412 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-076cc513-da64-4db6-b4b4-b078f22e9bea None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:11:22.667305 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:11:22.675342 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 293/580] 10.222.0.15 () {56 vars in 1109 bytes} [Thu Sep 15 12:11:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:11:22.676200 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076cc513-da64-4db6-b4b4-b078f22e9bea None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:22.725973 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8232be54-41f7-4e7a-b6d3-dcfed1d21679 None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:22.746687 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:11:22.761809 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03b7414-c397-47a8-a83f-65f03d4c196c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:22.762544 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03b7414-c397-47a8-a83f-65f03d4c196c None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:22.762916 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03b7414-c397-47a8-a83f-65f03d4c196c None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:22.763189 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a03b7414-c397-47a8-a83f-65f03d4c196c None admin] PATH_INFO: `/v3/domains` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:22.763847 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a03b7414-c397-47a8-a83f-65f03d4c196c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81449) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:11:22.764144 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a03b7414-c397-47a8-a83f-65f03d4c196c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:11:22.765589 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a03b7414-c397-47a8-a83f-65f03d4c196c None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:11:22.777652 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 288/581] 10.222.0.15 () {56 vars in 1109 bytes} [Thu Sep 15 12:11:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:11:22.778440 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03b7414-c397-47a8-a83f-65f03d4c196c None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:22.786036 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076cc513-da64-4db6-b4b4-b078f22e9bea None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:22.825860 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af10ea6b-dcbf-447f-b353-07d0c4b02b43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:22.826539 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af10ea6b-dcbf-447f-b353-07d0c4b02b43 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:22.826819 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af10ea6b-dcbf-447f-b353-07d0c4b02b43 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:22.827239 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af10ea6b-dcbf-447f-b353-07d0c4b02b43 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:22.838552 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af10ea6b-dcbf-447f-b353-07d0c4b02b43 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:11:22.840536 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af10ea6b-dcbf-447f-b353-07d0c4b02b43 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:11:22.852820 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application [None req-af10ea6b-dcbf-447f-b353-07d0c4b02b43 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 15 12:11:22.852820 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:11:22.852820 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:11:22.852820 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:11:22.852820 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:11:22.852820 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:11:22.852820 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:11:22.852820 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:11:22.852820 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:11:22.852820 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:11:22.852820 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:11:22.852820 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:11:22.852820 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 15 12:11:22.852820 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 15 12:11:22.852820 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 15 12:11:22.852820 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 15 12:11:22.853914 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:11:22.853914 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:11:22.853914 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 15 12:11:22.853914 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 12:11:22.853914 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 15 12:11:22.853914 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 12:11:22.853914 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:11:22.853914 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:11:22.853914 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:11:22.853914 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:11:22.853914 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:11:22.853914 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application with Lock( Sep 15 12:11:22.853914 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:11:22.853914 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:11:22.853914 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:11:22.853914 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:11:22.853914 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:11:22.853914 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:11:22.855384 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:11:22.855384 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:11:22.855384 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 15 12:11:22.855384 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 15 12:11:22.855384 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 15 12:11:22.855384 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 15 12:11:22.855384 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 15 12:11:22.855384 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 15 12:11:22.855384 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 15 12:11:22.855384 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 15 12:11:22.858566 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 294/582] 10.222.0.15 () {56 vars in 1099 bytes} [Thu Sep 15 12:11:22 2022] GET /identity/v3/users/cinder => generated 84 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:11:22.859317 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af10ea6b-dcbf-447f-b353-07d0c4b02b43 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:22.891239 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a03b7414-c397-47a8-a83f-65f03d4c196c None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:22.920312 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:11:22.930563 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3fc0ce6-abf1-4ae0-9d4c-0fefe8e9ac47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:22.931788 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fc0ce6-abf1-4ae0-9d4c-0fefe8e9ac47 None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:22.932082 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fc0ce6-abf1-4ae0-9d4c-0fefe8e9ac47 None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:22.932357 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fc0ce6-abf1-4ae0-9d4c-0fefe8e9ac47 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:22.942939 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3fc0ce6-abf1-4ae0-9d4c-0fefe8e9ac47 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:11:22.944384 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3fc0ce6-abf1-4ae0-9d4c-0fefe8e9ac47 None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:11:22.960388 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application [None req-b3fc0ce6-abf1-4ae0-9d4c-0fefe8e9ac47 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 12:11:22.960388 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:11:22.960388 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:11:22.960388 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:11:22.960388 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:11:22.960388 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:11:22.960388 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:11:22.960388 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:11:22.960388 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:11:22.960388 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:11:22.960388 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:11:22.960388 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:11:22.960388 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 12:11:22.960388 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 12:11:22.960388 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 12:11:22.960388 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 12:11:22.961389 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:11:22.961389 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:11:22.961389 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:11:22.961389 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:11:22.961389 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:11:22.961389 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:11:22.961389 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:11:22.961389 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application with Lock( Sep 15 12:11:22.961389 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:11:22.961389 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:11:22.961389 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:11:22.961389 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:11:22.961389 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:11:22.961389 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:11:22.961389 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:11:22.961389 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:11:22.961389 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 12:11:22.962291 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 12:11:22.962291 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 12:11:22.962291 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application Sep 15 12:11:22.962291 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 289/583] 10.222.0.15 () {56 vars in 1108 bytes} [Thu Sep 15 12:11:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:11:22.962291 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3fc0ce6-abf1-4ae0-9d4c-0fefe8e9ac47 None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:22.976319 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af10ea6b-dcbf-447f-b353-07d0c4b02b43 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:22.997775 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:11:23.015629 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e584ec-de5d-4d46-9b09-e15f8ded8a28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:23.016270 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e584ec-de5d-4d46-9b09-e15f8ded8a28 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:23.016558 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e584ec-de5d-4d46-9b09-e15f8ded8a28 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:23.016871 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e584ec-de5d-4d46-9b09-e15f8ded8a28 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:23.025414 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e584ec-de5d-4d46-9b09-e15f8ded8a28 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:11:23.026874 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e584ec-de5d-4d46-9b09-e15f8ded8a28 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:11:23.038500 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application [None req-c1e584ec-de5d-4d46-9b09-e15f8ded8a28 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 15 12:11:23.038500 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:11:23.038500 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:11:23.038500 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:11:23.038500 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:11:23.038500 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:11:23.038500 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:11:23.038500 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:11:23.038500 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:11:23.038500 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:11:23.038500 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:11:23.038500 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:11:23.038500 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 15 12:11:23.038500 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 15 12:11:23.038500 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 15 12:11:23.038500 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 15 12:11:23.039470 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:11:23.039470 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:11:23.039470 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 15 12:11:23.039470 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 12:11:23.039470 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 15 12:11:23.039470 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 12:11:23.039470 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:11:23.039470 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:11:23.039470 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:11:23.039470 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:11:23.039470 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:11:23.039470 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application with Lock( Sep 15 12:11:23.039470 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:11:23.039470 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:11:23.039470 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:11:23.039470 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:11:23.039470 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:11:23.039470 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:11:23.040426 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:11:23.040426 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:11:23.040426 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 15 12:11:23.040426 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 15 12:11:23.040426 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 15 12:11:23.040426 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 15 12:11:23.040426 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 15 12:11:23.040426 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 15 12:11:23.040426 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 15 12:11:23.040426 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 15 12:11:23.040426 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 295/584] 10.222.0.15 () {56 vars in 1102 bytes} [Thu Sep 15 12:11:22 2022] GET /identity/v3/users/neutron => generated 85 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:11:23.040426 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e584ec-de5d-4d46-9b09-e15f8ded8a28 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:23.075312 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3fc0ce6-abf1-4ae0-9d4c-0fefe8e9ac47 None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:23.111093 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6bfd1f-0142-4ff4-976d-e3d1bc89cd46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:23.111646 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6bfd1f-0142-4ff4-976d-e3d1bc89cd46 None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:23.111923 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6bfd1f-0142-4ff4-976d-e3d1bc89cd46 None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:23.112193 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6bfd1f-0142-4ff4-976d-e3d1bc89cd46 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:23.119999 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6bfd1f-0142-4ff4-976d-e3d1bc89cd46 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:11:23.121748 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6bfd1f-0142-4ff4-976d-e3d1bc89cd46 None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:11:23.133774 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application [None req-dd6bfd1f-0142-4ff4-976d-e3d1bc89cd46 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 15 12:11:23.133774 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:11:23.133774 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:11:23.133774 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:11:23.133774 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:11:23.133774 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:11:23.133774 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:11:23.133774 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:11:23.133774 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:11:23.133774 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:11:23.133774 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:11:23.133774 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:11:23.133774 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 15 12:11:23.133774 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 15 12:11:23.133774 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 15 12:11:23.133774 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 15 12:11:23.134751 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:11:23.134751 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:11:23.134751 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 15 12:11:23.134751 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 12:11:23.134751 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 15 12:11:23.134751 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 12:11:23.134751 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:11:23.134751 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:11:23.134751 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:11:23.134751 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:11:23.134751 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:11:23.134751 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application with Lock( Sep 15 12:11:23.134751 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:11:23.134751 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:11:23.134751 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:11:23.134751 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:11:23.134751 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:11:23.134751 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:11:23.135687 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:11:23.135687 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:11:23.135687 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 15 12:11:23.135687 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 15 12:11:23.135687 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 15 12:11:23.135687 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 15 12:11:23.135687 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 15 12:11:23.135687 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 15 12:11:23.135687 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 15 12:11:23.135687 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application Sep 15 12:11:23.136574 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 290/585] 10.222.0.15 () {56 vars in 1099 bytes} [Thu Sep 15 12:11:22 2022] GET /identity/v3/users/glance => generated 84 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:11:23.137421 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6bfd1f-0142-4ff4-976d-e3d1bc89cd46 None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:23.151960 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e584ec-de5d-4d46-9b09-e15f8ded8a28 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:23.174090 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:11:23.190372 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd98236-0b42-495b-9ed1-191562cd028c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:23.190989 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd98236-0b42-495b-9ed1-191562cd028c None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:23.191269 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd98236-0b42-495b-9ed1-191562cd028c None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:23.191541 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd98236-0b42-495b-9ed1-191562cd028c None admin] PATH_INFO: `/v3/users` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:23.192271 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd98236-0b42-495b-9ed1-191562cd028c None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:11:23.193176 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd98236-0b42-495b-9ed1-191562cd028c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:11:23.194589 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd98236-0b42-495b-9ed1-191562cd028c None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:11:23.212477 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 296/586] 10.222.0.15 () {56 vars in 1137 bytes} [Thu Sep 15 12:11:23 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:11:23.213331 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd98236-0b42-495b-9ed1-191562cd028c None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:23.245449 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:11:23.257962 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6bfd1f-0142-4ff4-976d-e3d1bc89cd46 None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:23.294147 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200f56da-c21d-44ee-a04f-49a48a2cf8ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:23.294753 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200f56da-c21d-44ee-a04f-49a48a2cf8ba None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:23.295027 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200f56da-c21d-44ee-a04f-49a48a2cf8ba None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:23.295299 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200f56da-c21d-44ee-a04f-49a48a2cf8ba None admin] PATH_INFO: `/v3/domains` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:23.295996 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-200f56da-c21d-44ee-a04f-49a48a2cf8ba None admin] RBAC: Adding query filter params (name=Default) {{(pid=81449) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:11:23.296301 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-200f56da-c21d-44ee-a04f-49a48a2cf8ba None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:11:23.297677 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-200f56da-c21d-44ee-a04f-49a48a2cf8ba None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:11:23.309229 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 291/587] 10.222.0.15 () {56 vars in 1109 bytes} [Thu Sep 15 12:11:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:11:23.314472 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200f56da-c21d-44ee-a04f-49a48a2cf8ba None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:23.315801 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd98236-0b42-495b-9ed1-191562cd028c None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:23.350551 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05794271-4681-4b1c-ad65-36349c0f0301 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:23.351321 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05794271-4681-4b1c-ad65-36349c0f0301 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:23.351594 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05794271-4681-4b1c-ad65-36349c0f0301 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:23.352606 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05794271-4681-4b1c-ad65-36349c0f0301 None admin] PATH_INFO: `/v3/users` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:23.353357 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05794271-4681-4b1c-ad65-36349c0f0301 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:11:23.353667 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05794271-4681-4b1c-ad65-36349c0f0301 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:11:23.355330 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05794271-4681-4b1c-ad65-36349c0f0301 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:11:23.377931 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 297/588] 10.222.0.15 () {56 vars in 1139 bytes} [Thu Sep 15 12:11:23 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:11:23.377931 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05794271-4681-4b1c-ad65-36349c0f0301 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:23.412364 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:11:23.423411 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200f56da-c21d-44ee-a04f-49a48a2cf8ba None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:23.461466 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef92224-f3df-4e72-b775-07460e719cba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:23.462098 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef92224-f3df-4e72-b775-07460e719cba None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:23.462370 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef92224-f3df-4e72-b775-07460e719cba None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:23.462635 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef92224-f3df-4e72-b775-07460e719cba None admin] PATH_INFO: `/v3/users` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:23.463372 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef92224-f3df-4e72-b775-07460e719cba None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81449) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:11:23.463671 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef92224-f3df-4e72-b775-07460e719cba None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:11:23.465148 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef92224-f3df-4e72-b775-07460e719cba None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:11:23.487798 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 292/589] 10.222.0.15 () {56 vars in 1137 bytes} [Thu Sep 15 12:11:23 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:11:23.487798 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef92224-f3df-4e72-b775-07460e719cba None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:23.488092 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05794271-4681-4b1c-ad65-36349c0f0301 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:23.498341 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:11:23.525954 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989420a2-369e-456e-8024-b9d6e25898df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:23.526589 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989420a2-369e-456e-8024-b9d6e25898df None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:23.526869 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989420a2-369e-456e-8024-b9d6e25898df None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:23.527552 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989420a2-369e-456e-8024-b9d6e25898df None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:23.539466 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989420a2-369e-456e-8024-b9d6e25898df None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:11:23.540988 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989420a2-369e-456e-8024-b9d6e25898df None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:11:23.559506 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application [None req-989420a2-369e-456e-8024-b9d6e25898df None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 12:11:23.559506 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:11:23.559506 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:11:23.559506 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:11:23.559506 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:11:23.559506 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:11:23.559506 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:11:23.559506 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:11:23.559506 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:11:23.559506 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:11:23.559506 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:11:23.559506 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:11:23.559506 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 12:11:23.559506 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 12:11:23.559506 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 12:11:23.559506 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 12:11:23.560518 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:11:23.560518 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:11:23.560518 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:11:23.560518 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:11:23.560518 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:11:23.560518 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:11:23.560518 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:11:23.560518 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application with Lock( Sep 15 12:11:23.560518 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:11:23.560518 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:11:23.560518 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:11:23.560518 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:11:23.560518 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:11:23.560518 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:11:23.560518 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:11:23.560518 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:11:23.560518 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 12:11:23.561489 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 12:11:23.561489 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 12:11:23.561489 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 15 12:11:23.561489 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 298/590] 10.222.0.15 () {56 vars in 1108 bytes} [Thu Sep 15 12:11:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:11:23.562299 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989420a2-369e-456e-8024-b9d6e25898df None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:23.590382 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef92224-f3df-4e72-b775-07460e719cba None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:23.623849 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a006dfc-5446-457f-8742-f2b4327ecd7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:23.624594 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a006dfc-5446-457f-8742-f2b4327ecd7b None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:23.624963 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a006dfc-5446-457f-8742-f2b4327ecd7b None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:23.625240 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a006dfc-5446-457f-8742-f2b4327ecd7b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:23.632482 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a006dfc-5446-457f-8742-f2b4327ecd7b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:11:23.633894 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a006dfc-5446-457f-8742-f2b4327ecd7b None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:11:23.644669 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application [None req-7a006dfc-5446-457f-8742-f2b4327ecd7b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 12:11:23.644669 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:11:23.644669 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:11:23.644669 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:11:23.644669 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:11:23.644669 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:11:23.644669 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:11:23.644669 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:11:23.644669 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:11:23.644669 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:11:23.644669 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:11:23.644669 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:11:23.644669 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 15 12:11:23.644669 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 15 12:11:23.644669 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 15 12:11:23.645780 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 15 12:11:23.645780 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:11:23.645780 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:11:23.645780 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:11:23.645780 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:11:23.645780 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:11:23.645780 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:11:23.645780 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:11:23.645780 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application with Lock( Sep 15 12:11:23.645780 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:11:23.645780 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:11:23.645780 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:11:23.645780 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:11:23.645780 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:11:23.645780 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:11:23.645780 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:11:23.645780 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:11:23.646855 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 15 12:11:23.646855 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 15 12:11:23.646855 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 15 12:11:23.646855 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 15 12:11:23.646855 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 15 12:11:23.646855 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 15 12:11:23.646855 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 12:11:23.646855 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application Sep 15 12:11:23.646855 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 293/591] 10.222.0.15 () {56 vars in 1111 bytes} [Thu Sep 15 12:11:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:11:23.646855 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a006dfc-5446-457f-8742-f2b4327ecd7b None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:23.652889 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:11:23.665077 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989420a2-369e-456e-8024-b9d6e25898df None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:23.698375 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff8b07a-1f93-4efc-a394-1364fbd11980 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:23.699004 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff8b07a-1f93-4efc-a394-1364fbd11980 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:23.699279 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff8b07a-1f93-4efc-a394-1364fbd11980 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:23.699548 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff8b07a-1f93-4efc-a394-1364fbd11980 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:23.710498 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff8b07a-1f93-4efc-a394-1364fbd11980 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:11:23.711926 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff8b07a-1f93-4efc-a394-1364fbd11980 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:11:23.715492 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:11:23.726773 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application [None req-4ff8b07a-1f93-4efc-a394-1364fbd11980 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 12:11:23.726773 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:11:23.726773 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:11:23.726773 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:11:23.726773 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:11:23.726773 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:11:23.726773 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:11:23.726773 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:11:23.726773 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:11:23.726773 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:11:23.726773 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:11:23.726773 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:11:23.726773 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 12:11:23.726773 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 12:11:23.726773 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 12:11:23.726773 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 12:11:23.727909 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:11:23.727909 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:11:23.727909 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:11:23.727909 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:11:23.727909 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:11:23.727909 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:11:23.727909 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:11:23.727909 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application with Lock( Sep 15 12:11:23.727909 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:11:23.727909 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:11:23.727909 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:11:23.727909 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:11:23.727909 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:11:23.727909 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:11:23.727909 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:11:23.727909 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:11:23.727909 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 12:11:23.729025 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 12:11:23.729025 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 12:11:23.729025 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 15 12:11:23.729025 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 299/592] 10.222.0.15 () {56 vars in 1108 bytes} [Thu Sep 15 12:11:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:11:23.729025 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62bb1d8-9bc2-4b20-9f40-bf2879eab46e None None] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:23.729324 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62bb1d8-9bc2-4b20-9f40-bf2879eab46e None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:23.729550 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62bb1d8-9bc2-4b20-9f40-bf2879eab46e None None] PATH_INFO: `/` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:23.732943 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 300/593] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 12:11:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:11:23.732943 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62bb1d8-9bc2-4b20-9f40-bf2879eab46e None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:23.753003 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a006dfc-5446-457f-8742-f2b4327ecd7b None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:23.785790 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387ac070-c5b7-473d-8d9a-b460b75d79e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:23.786961 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387ac070-c5b7-473d-8d9a-b460b75d79e0 None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:23.787240 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387ac070-c5b7-473d-8d9a-b460b75d79e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:23.787506 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387ac070-c5b7-473d-8d9a-b460b75d79e0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:23.797626 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387ac070-c5b7-473d-8d9a-b460b75d79e0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:11:23.799062 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387ac070-c5b7-473d-8d9a-b460b75d79e0 None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:11:23.813326 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application [None req-387ac070-c5b7-473d-8d9a-b460b75d79e0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 12:11:23.813326 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:11:23.813326 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:11:23.813326 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:11:23.813326 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:11:23.813326 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:11:23.813326 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:11:23.813326 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:11:23.813326 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:11:23.813326 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:11:23.813326 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:11:23.813326 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:11:23.813326 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 12:11:23.813326 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 12:11:23.813326 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 12:11:23.813326 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 12:11:23.814444 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:11:23.814444 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:11:23.814444 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:11:23.814444 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:11:23.814444 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:11:23.814444 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:11:23.814444 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:11:23.814444 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application with Lock( Sep 15 12:11:23.814444 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:11:23.814444 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:11:23.814444 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:11:23.814444 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:11:23.814444 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:11:23.814444 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:11:23.814444 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:11:23.814444 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:11:23.814444 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 12:11:23.815528 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 12:11:23.815528 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 12:11:23.815528 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application Sep 15 12:11:23.815528 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 294/594] 10.222.0.15 () {56 vars in 1108 bytes} [Thu Sep 15 12:11:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:11:23.815528 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387ac070-c5b7-473d-8d9a-b460b75d79e0 None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:23.837783 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62bb1d8-9bc2-4b20-9f40-bf2879eab46e None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:23.870062 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8789033-1e08-43b3-9507-5866b32af709 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:23.870669 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8789033-1e08-43b3-9507-5866b32af709 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:23.870947 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8789033-1e08-43b3-9507-5866b32af709 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:23.871217 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8789033-1e08-43b3-9507-5866b32af709 None admin] PATH_INFO: `/v3/domains` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:23.871843 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8789033-1e08-43b3-9507-5866b32af709 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:11:23.872143 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8789033-1e08-43b3-9507-5866b32af709 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:11:23.873517 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8789033-1e08-43b3-9507-5866b32af709 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:11:23.882469 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:11:23.886891 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 301/595] 10.222.0.15 () {56 vars in 1109 bytes} [Thu Sep 15 12:11:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:11:23.887630 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8789033-1e08-43b3-9507-5866b32af709 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:23.913910 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387ac070-c5b7-473d-8d9a-b460b75d79e0 None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:23.936156 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:11:23.949876 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90c3d5e-7c37-4f0f-a2b6-b224bd3f5284 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:23.950483 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90c3d5e-7c37-4f0f-a2b6-b224bd3f5284 None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:23.950760 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90c3d5e-7c37-4f0f-a2b6-b224bd3f5284 None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:23.951028 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90c3d5e-7c37-4f0f-a2b6-b224bd3f5284 None admin] PATH_INFO: `/v3/projects` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:23.951739 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b90c3d5e-7c37-4f0f-a2b6-b224bd3f5284 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81449) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:11:23.952041 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b90c3d5e-7c37-4f0f-a2b6-b224bd3f5284 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:11:23.953506 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b90c3d5e-7c37-4f0f-a2b6-b224bd3f5284 None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:11:23.968013 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 295/596] 10.222.0.15 () {56 vars in 1148 bytes} [Thu Sep 15 12:11:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:11:23.969223 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f929b904-7ecf-4e03-b686-e47597bc9b82 None None] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:23.969501 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f929b904-7ecf-4e03-b686-e47597bc9b82 None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:23.969765 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f929b904-7ecf-4e03-b686-e47597bc9b82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:23.987388 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.auth.core [None req-f929b904-7ecf-4e03-b686-e47597bc9b82 None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81449) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:11:23.988553 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-f929b904-7ecf-4e03-b686-e47597bc9b82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:11:23.994060 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8789033-1e08-43b3-9507-5866b32af709 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:24.027574 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b75e6d8-3aa1-4e45-baa6-3dd16b98ccfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:24.028187 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b75e6d8-3aa1-4e45-baa6-3dd16b98ccfa None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:24.028467 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b75e6d8-3aa1-4e45-baa6-3dd16b98ccfa None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:24.028739 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b75e6d8-3aa1-4e45-baa6-3dd16b98ccfa None admin] PATH_INFO: `/v3/domains` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:24.029405 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b75e6d8-3aa1-4e45-baa6-3dd16b98ccfa None admin] RBAC: Adding query filter params (name=Default) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:11:24.029709 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b75e6d8-3aa1-4e45-baa6-3dd16b98ccfa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:11:24.031093 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b75e6d8-3aa1-4e45-baa6-3dd16b98ccfa None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:11:24.042818 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 302/597] 10.222.0.15 () {56 vars in 1109 bytes} [Thu Sep 15 12:11:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:11:24.043425 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b75e6d8-3aa1-4e45-baa6-3dd16b98ccfa None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:24.056807 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 296/598] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 12:11:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 12:11:24.057638 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f929b904-7ecf-4e03-b686-e47597bc9b82 None None] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:24.105646 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:11:24.148600 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b75e6d8-3aa1-4e45-baa6-3dd16b98ccfa None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:24.157129 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f929b904-7ecf-4e03-b686-e47597bc9b82 None None] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:24.170270 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:11:24.187391 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f64766-f130-4634-9c99-55c88bc5ccc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:24.188232 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f64766-f130-4634-9c99-55c88bc5ccc5 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:24.188669 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f64766-f130-4634-9c99-55c88bc5ccc5 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:24.189180 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f64766-f130-4634-9c99-55c88bc5ccc5 None admin] PATH_INFO: `/v3/domains` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:24.190127 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f64766-f130-4634-9c99-55c88bc5ccc5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:11:24.190670 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f64766-f130-4634-9c99-55c88bc5ccc5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:11:24.191705 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01854db2-2180-4f5c-bcbf-0313337aa7cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:24.192335 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01854db2-2180-4f5c-bcbf-0313337aa7cf None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:24.193241 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01854db2-2180-4f5c-bcbf-0313337aa7cf None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:24.193465 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f64766-f130-4634-9c99-55c88bc5ccc5 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:11:24.193592 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01854db2-2180-4f5c-bcbf-0313337aa7cf None admin] PATH_INFO: `/v3/projects/service` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:24.200029 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01854db2-2180-4f5c-bcbf-0313337aa7cf None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:11:24.201477 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01854db2-2180-4f5c-bcbf-0313337aa7cf None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:11:24.206473 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 303/599] 10.222.0.15 () {56 vars in 1109 bytes} [Thu Sep 15 12:11:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:11:24.207431 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f64766-f130-4634-9c99-55c88bc5ccc5 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:24.216946 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application [None req-01854db2-2180-4f5c-bcbf-0313337aa7cf None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 12:11:24.216946 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:11:24.216946 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:11:24.216946 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:11:24.216946 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:11:24.216946 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:11:24.216946 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:11:24.216946 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:11:24.216946 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:11:24.216946 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:11:24.216946 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:11:24.216946 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:11:24.216946 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 15 12:11:24.216946 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 15 12:11:24.216946 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 15 12:11:24.217945 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 15 12:11:24.217945 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:11:24.217945 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:11:24.217945 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:11:24.217945 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:11:24.217945 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:11:24.217945 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:11:24.217945 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:11:24.217945 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application with Lock( Sep 15 12:11:24.217945 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:11:24.217945 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:11:24.217945 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:11:24.217945 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:11:24.217945 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:11:24.217945 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:11:24.217945 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:11:24.217945 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:11:24.218862 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 15 12:11:24.218862 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 15 12:11:24.218862 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 15 12:11:24.218862 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 15 12:11:24.218862 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 15 12:11:24.218862 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 15 12:11:24.218862 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 12:11:24.218862 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application Sep 15 12:11:24.218862 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 297/600] 10.222.0.15 () {56 vars in 1111 bytes} [Thu Sep 15 12:11:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:11:24.219726 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01854db2-2180-4f5c-bcbf-0313337aa7cf None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:24.321002 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f64766-f130-4634-9c99-55c88bc5ccc5 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:24.325456 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01854db2-2180-4f5c-bcbf-0313337aa7cf None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:24.341516 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:11:24.356693 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25934f65-5ea2-4a91-9b40-53739c181724 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:24.357418 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25934f65-5ea2-4a91-9b40-53739c181724 None admin] REQUEST_METHOD: `PUT` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:24.357703 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25934f65-5ea2-4a91-9b40-53739c181724 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:24.357975 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25934f65-5ea2-4a91-9b40-53739c181724 None admin] PATH_INFO: `/v3/projects/540a32ef88764e8c8d9201ae22f6ddaf/users/eface89e261840a2abb4206be26455c7/roles/e1cecc29e028497f9cdaa04268c6a6ec` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:24.358240 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed182ff7-d204-4eab-b0d4-65ccd15b164d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:24.358862 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed182ff7-d204-4eab-b0d4-65ccd15b164d None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:24.359136 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed182ff7-d204-4eab-b0d4-65ccd15b164d None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:24.359400 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed182ff7-d204-4eab-b0d4-65ccd15b164d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:24.366336 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed182ff7-d204-4eab-b0d4-65ccd15b164d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:11:24.367752 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed182ff7-d204-4eab-b0d4-65ccd15b164d None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:11:24.372548 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25934f65-5ea2-4a91-9b40-53739c181724 None admin] RBAC: Authorizing `identity:create_grant(project_id=540a32ef88764e8c8d9201ae22f6ddaf, user_id=eface89e261840a2abb4206be26455c7, role_id=e1cecc29e028497f9cdaa04268c6a6ec)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:11:24.374005 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25934f65-5ea2-4a91-9b40-53739c181724 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:11:24.378607 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application [None req-ed182ff7-d204-4eab-b0d4-65ccd15b164d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 12:11:24.378607 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:11:24.378607 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:11:24.378607 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:11:24.378607 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:11:24.378607 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:11:24.378607 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:11:24.378607 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:11:24.378607 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:11:24.378607 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:11:24.378607 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:11:24.378607 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:11:24.378607 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 15 12:11:24.378607 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 15 12:11:24.378607 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 15 12:11:24.379660 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 15 12:11:24.379660 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:11:24.379660 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:11:24.379660 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:11:24.379660 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:11:24.379660 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:11:24.379660 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:11:24.379660 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:11:24.379660 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application with Lock( Sep 15 12:11:24.379660 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:11:24.379660 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:11:24.379660 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:11:24.379660 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:11:24.379660 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:11:24.379660 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:11:24.379660 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:11:24.379660 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:11:24.381618 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 15 12:11:24.381618 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 15 12:11:24.381618 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 15 12:11:24.381618 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 15 12:11:24.381618 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 15 12:11:24.381618 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 15 12:11:24.381618 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 12:11:24.381618 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application Sep 15 12:11:24.381618 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 298/601] 10.222.0.15 () {56 vars in 1111 bytes} [Thu Sep 15 12:11:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:11:24.381618 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed182ff7-d204-4eab-b0d4-65ccd15b164d None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:24.391403 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 304/602] 10.222.0.15 () {58 vars in 1439 bytes} [Thu Sep 15 12:11:24 2022] PUT /identity/v3/projects/540a32ef88764e8c8d9201ae22f6ddaf/users/eface89e261840a2abb4206be26455c7/roles/e1cecc29e028497f9cdaa04268c6a6ec => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:11:24.392218 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25934f65-5ea2-4a91-9b40-53739c181724 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:24.412833 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:11:24.493454 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed182ff7-d204-4eab-b0d4-65ccd15b164d None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:24.516025 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25934f65-5ea2-4a91-9b40-53739c181724 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:24.531672 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe01780-89b2-460f-a387-f227120f4550 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:24.532328 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe01780-89b2-460f-a387-f227120f4550 None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:24.532604 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe01780-89b2-460f-a387-f227120f4550 None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:24.532913 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe01780-89b2-460f-a387-f227120f4550 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:24.541235 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe01780-89b2-460f-a387-f227120f4550 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:11:24.542961 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe01780-89b2-460f-a387-f227120f4550 None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:11:24.553431 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e2dbec-b732-41d5-ac0b-22eea040d658 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:24.554055 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e2dbec-b732-41d5-ac0b-22eea040d658 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:24.554336 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e2dbec-b732-41d5-ac0b-22eea040d658 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:24.554605 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e2dbec-b732-41d5-ac0b-22eea040d658 None admin] PATH_INFO: `/v3/projects` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:24.555328 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e2dbec-b732-41d5-ac0b-22eea040d658 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:11:24.555634 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e2dbec-b732-41d5-ac0b-22eea040d658 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:11:24.556007 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application [None req-dfe01780-89b2-460f-a387-f227120f4550 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 12:11:24.556007 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:11:24.556007 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:11:24.556007 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:11:24.556007 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:11:24.556007 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:11:24.556007 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:11:24.556007 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:11:24.556007 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:11:24.556007 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:11:24.556007 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:11:24.556007 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:11:24.556007 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 15 12:11:24.556007 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 15 12:11:24.556007 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 15 12:11:24.557127 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 15 12:11:24.557127 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:11:24.557127 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:11:24.557127 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:11:24.557127 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:11:24.557127 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:11:24.557127 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:11:24.557127 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:11:24.557127 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application with Lock( Sep 15 12:11:24.557127 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:11:24.557127 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:11:24.557127 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:11:24.557127 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:11:24.557127 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:11:24.557127 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:11:24.557127 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:11:24.557127 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:11:24.558056 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 15 12:11:24.558056 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 15 12:11:24.558056 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 15 12:11:24.558056 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 15 12:11:24.558056 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 15 12:11:24.558056 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 15 12:11:24.558056 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 12:11:24.558056 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application Sep 15 12:11:24.558524 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e2dbec-b732-41d5-ac0b-22eea040d658 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:11:24.561974 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 299/603] 10.222.0.15 () {56 vars in 1111 bytes} [Thu Sep 15 12:11:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:11:24.562872 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe01780-89b2-460f-a387-f227120f4550 None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:24.574655 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 305/604] 10.222.0.15 () {56 vars in 1148 bytes} [Thu Sep 15 12:11:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:11:24.575740 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e2dbec-b732-41d5-ac0b-22eea040d658 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:24.609774 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:11:24.663306 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:11:24.673775 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe01780-89b2-460f-a387-f227120f4550 None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:24.688520 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e2dbec-b732-41d5-ac0b-22eea040d658 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:24.709291 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1fe5c1d-e824-4ea9-be94-9f684c8fe996 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:24.709963 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fe5c1d-e824-4ea9-be94-9f684c8fe996 None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:24.710268 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fe5c1d-e824-4ea9-be94-9f684c8fe996 None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:24.710522 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1fe5c1d-e824-4ea9-be94-9f684c8fe996 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:24.720788 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1fe5c1d-e824-4ea9-be94-9f684c8fe996 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 12:11:24.723098 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ee6666-c0b9-48fe-84c8-655640db4f9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:24.724261 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ee6666-c0b9-48fe-84c8-655640db4f9e None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:24.724542 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ee6666-c0b9-48fe-84c8-655640db4f9e None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:24.724835 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ee6666-c0b9-48fe-84c8-655640db4f9e None admin] PATH_INFO: `/v3/projects` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:24.725515 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ee6666-c0b9-48fe-84c8-655640db4f9e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:11:24.725815 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ee6666-c0b9-48fe-84c8-655640db4f9e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:11:24.726563 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application [None req-a1fe5c1d-e824-4ea9-be94-9f684c8fe996 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 15 12:11:24.726563 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:11:24.726563 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:11:24.726563 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:11:24.726563 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:11:24.726563 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:11:24.726563 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:11:24.726563 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:11:24.726563 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:11:24.726563 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:11:24.726563 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:11:24.726563 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:11:24.726563 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 12:11:24.726563 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 12:11:24.726563 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 12:11:24.726563 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 12:11:24.727456 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 12:11:24.727456 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 12:11:24.727456 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 12:11:24.727456 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 12:11:24.727456 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:11:24.727456 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:11:24.727456 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:11:24.727456 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:11:24.727456 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:11:24.727456 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:11:24.727456 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:11:24.727456 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application with Lock( Sep 15 12:11:24.727456 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:11:24.727456 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:11:24.727456 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:11:24.727456 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:11:24.729941 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:11:24.729941 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:11:24.729941 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:11:24.729941 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:11:24.729941 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 12:11:24.729941 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 12:11:24.729941 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 12:11:24.729941 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 12:11:24.729941 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 12:11:24.729941 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 12:11:24.729941 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 15 12:11:24.729941 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application Sep 15 12:11:24.730614 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ee6666-c0b9-48fe-84c8-655640db4f9e None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:11:24.730687 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 300/605] 10.222.0.15 () {56 vars in 1102 bytes} [Thu Sep 15 12:11:24 2022] GET /identity/v3/roles/service => generated 85 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:11:24.730687 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1fe5c1d-e824-4ea9-be94-9f684c8fe996 None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:24.744975 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 306/606] 10.222.0.15 () {56 vars in 1148 bytes} [Thu Sep 15 12:11:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:11:24.745859 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ee6666-c0b9-48fe-84c8-655640db4f9e None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:24.841780 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1fe5c1d-e824-4ea9-be94-9f684c8fe996 None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:24.847853 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:11:24.860062 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ee6666-c0b9-48fe-84c8-655640db4f9e None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:24.879612 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193d92ed-f21a-4e14-bc15-90d7f642fab0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:24.880258 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193d92ed-f21a-4e14-bc15-90d7f642fab0 None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:24.880542 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193d92ed-f21a-4e14-bc15-90d7f642fab0 None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:24.880887 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193d92ed-f21a-4e14-bc15-90d7f642fab0 None admin] PATH_INFO: `/v3/projects` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:24.881610 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-193d92ed-f21a-4e14-bc15-90d7f642fab0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81449) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:11:24.881922 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-193d92ed-f21a-4e14-bc15-90d7f642fab0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:11:24.883426 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-193d92ed-f21a-4e14-bc15-90d7f642fab0 None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:11:24.895450 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 301/607] 10.222.0.15 () {56 vars in 1148 bytes} [Thu Sep 15 12:11:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:11:24.897117 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193d92ed-f21a-4e14-bc15-90d7f642fab0 None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:24.907414 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6e0895-bd41-4ea9-92f1-3699907d55e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:24.908572 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6e0895-bd41-4ea9-92f1-3699907d55e3 None admin] REQUEST_METHOD: `PUT` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:24.908952 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6e0895-bd41-4ea9-92f1-3699907d55e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:24.909293 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6e0895-bd41-4ea9-92f1-3699907d55e3 None admin] PATH_INFO: `/v3/projects/540a32ef88764e8c8d9201ae22f6ddaf/users/c8191d3ccb874529b32f8d0026db629d/roles/e1cecc29e028497f9cdaa04268c6a6ec` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:24.915910 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:11:24.928408 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6e0895-bd41-4ea9-92f1-3699907d55e3 None admin] RBAC: Authorizing `identity:create_grant(project_id=540a32ef88764e8c8d9201ae22f6ddaf, user_id=c8191d3ccb874529b32f8d0026db629d, role_id=e1cecc29e028497f9cdaa04268c6a6ec)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:11:24.930415 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6e0895-bd41-4ea9-92f1-3699907d55e3 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:11:24.956784 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 307/608] 10.222.0.15 () {58 vars in 1439 bytes} [Thu Sep 15 12:11:24 2022] PUT /identity/v3/projects/540a32ef88764e8c8d9201ae22f6ddaf/users/c8191d3ccb874529b32f8d0026db629d/roles/e1cecc29e028497f9cdaa04268c6a6ec => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:11:24.956784 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6e0895-bd41-4ea9-92f1-3699907d55e3 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:25.035153 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193d92ed-f21a-4e14-bc15-90d7f642fab0 None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:25.075198 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce01de9-0861-4aae-9be8-47988c048329 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:25.075871 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce01de9-0861-4aae-9be8-47988c048329 None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:25.076155 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce01de9-0861-4aae-9be8-47988c048329 None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:25.076476 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce01de9-0861-4aae-9be8-47988c048329 None admin] PATH_INFO: `/v3/roles` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:25.077255 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce01de9-0861-4aae-9be8-47988c048329 None admin] RBAC: Adding query filter params (name=service) {{(pid=81449) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:11:25.077558 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce01de9-0861-4aae-9be8-47988c048329 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:11:25.079038 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce01de9-0861-4aae-9be8-47988c048329 None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:11:25.093589 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 302/609] 10.222.0.15 () {56 vars in 1103 bytes} [Thu Sep 15 12:11:24 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:11:25.094636 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce01de9-0861-4aae-9be8-47988c048329 None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:25.097963 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6e0895-bd41-4ea9-92f1-3699907d55e3 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:25.150409 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ed7235-1008-4297-9311-ec27bf59ed4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:25.151108 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ed7235-1008-4297-9311-ec27bf59ed4d None admin] REQUEST_METHOD: `PUT` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:25.151390 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ed7235-1008-4297-9311-ec27bf59ed4d None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:25.151663 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ed7235-1008-4297-9311-ec27bf59ed4d None admin] PATH_INFO: `/v3/projects/540a32ef88764e8c8d9201ae22f6ddaf/users/bfbb4d6809064f9cb9e06626ed7d6814/roles/e1cecc29e028497f9cdaa04268c6a6ec` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:25.164553 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:11:25.169460 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ed7235-1008-4297-9311-ec27bf59ed4d None admin] RBAC: Authorizing `identity:create_grant(project_id=540a32ef88764e8c8d9201ae22f6ddaf, user_id=bfbb4d6809064f9cb9e06626ed7d6814, role_id=e1cecc29e028497f9cdaa04268c6a6ec)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:11:25.171415 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ed7235-1008-4297-9311-ec27bf59ed4d None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:11:25.190315 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:11:25.190937 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 308/610] 10.222.0.15 () {58 vars in 1439 bytes} [Thu Sep 15 12:11:24 2022] PUT /identity/v3/projects/540a32ef88764e8c8d9201ae22f6ddaf/users/bfbb4d6809064f9cb9e06626ed7d6814/roles/e1cecc29e028497f9cdaa04268c6a6ec => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:11:25.191638 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ed7235-1008-4297-9311-ec27bf59ed4d None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:25.228517 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce01de9-0861-4aae-9be8-47988c048329 None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:25.275986 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76fd054-f646-4b18-ae28-634d53db63bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:25.277470 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76fd054-f646-4b18-ae28-634d53db63bc None admin] REQUEST_METHOD: `PUT` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:25.277902 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76fd054-f646-4b18-ae28-634d53db63bc None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:25.278276 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76fd054-f646-4b18-ae28-634d53db63bc None admin] PATH_INFO: `/v3/projects/540a32ef88764e8c8d9201ae22f6ddaf/users/3aed8f78903f44cda9e5254aa46cf2b7/roles/e1cecc29e028497f9cdaa04268c6a6ec` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:25.303260 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76fd054-f646-4b18-ae28-634d53db63bc None admin] RBAC: Authorizing `identity:create_grant(project_id=540a32ef88764e8c8d9201ae22f6ddaf, user_id=3aed8f78903f44cda9e5254aa46cf2b7, role_id=e1cecc29e028497f9cdaa04268c6a6ec)` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:11:25.305230 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76fd054-f646-4b18-ae28-634d53db63bc None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:11:25.305715 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ed7235-1008-4297-9311-ec27bf59ed4d None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:25.325506 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 303/611] 10.222.0.15 () {58 vars in 1439 bytes} [Thu Sep 15 12:11:25 2022] PUT /identity/v3/projects/540a32ef88764e8c8d9201ae22f6ddaf/users/3aed8f78903f44cda9e5254aa46cf2b7/roles/e1cecc29e028497f9cdaa04268c6a6ec => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:11:25.355542 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff85029-20d0-4bf4-8ed5-84dfacf0cbb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:25.356211 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff85029-20d0-4bf4-8ed5-84dfacf0cbb8 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:25.356486 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff85029-20d0-4bf4-8ed5-84dfacf0cbb8 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:25.356865 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff85029-20d0-4bf4-8ed5-84dfacf0cbb8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:25.370697 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff85029-20d0-4bf4-8ed5-84dfacf0cbb8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:11:25.372215 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff85029-20d0-4bf4-8ed5-84dfacf0cbb8 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:11:25.390261 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application [None req-eff85029-20d0-4bf4-8ed5-84dfacf0cbb8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 12:11:25.390261 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:11:25.390261 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:11:25.390261 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:11:25.390261 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:11:25.390261 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:11:25.390261 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:11:25.390261 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:11:25.390261 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:11:25.390261 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:11:25.390261 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:11:25.390261 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:11:25.390261 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 12:11:25.390261 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 12:11:25.390261 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 12:11:25.390261 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 12:11:25.391346 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:11:25.391346 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:11:25.391346 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:11:25.391346 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:11:25.391346 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:11:25.391346 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:11:25.391346 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:11:25.391346 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application with Lock( Sep 15 12:11:25.391346 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:11:25.391346 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:11:25.391346 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:11:25.391346 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:11:25.391346 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:11:25.391346 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:11:25.391346 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:11:25.391346 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:11:25.391346 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 12:11:25.392258 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 12:11:25.392258 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 12:11:25.392258 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 15 12:11:25.392258 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 309/612] 10.222.0.15 () {56 vars in 1108 bytes} [Thu Sep 15 12:11:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:11:25.398503 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76fd054-f646-4b18-ae28-634d53db63bc None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:25.523107 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76fd054-f646-4b18-ae28-634d53db63bc None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:25.550326 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:11:25.563074 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a43792c-a4fd-4a31-8b8e-04c49c2872a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:25.563924 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a43792c-a4fd-4a31-8b8e-04c49c2872a0 None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:25.564379 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a43792c-a4fd-4a31-8b8e-04c49c2872a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:25.564841 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a43792c-a4fd-4a31-8b8e-04c49c2872a0 None admin] PATH_INFO: `/v3/domains` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:25.565652 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a43792c-a4fd-4a31-8b8e-04c49c2872a0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81449) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:11:25.566104 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a43792c-a4fd-4a31-8b8e-04c49c2872a0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:11:25.568167 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a43792c-a4fd-4a31-8b8e-04c49c2872a0 None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:11:25.580424 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 304/613] 10.222.0.15 () {56 vars in 1109 bytes} [Thu Sep 15 12:11:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:11:25.586976 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff85029-20d0-4bf4-8ed5-84dfacf0cbb8 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:25.637049 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:11:25.702030 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff85029-20d0-4bf4-8ed5-84dfacf0cbb8 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:25.742178 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fdb5bf6-bc1a-49db-88ac-efde98cb2539 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:25.743101 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdb5bf6-bc1a-49db-88ac-efde98cb2539 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:25.743525 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdb5bf6-bc1a-49db-88ac-efde98cb2539 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:25.743933 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdb5bf6-bc1a-49db-88ac-efde98cb2539 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:25.752146 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fdb5bf6-bc1a-49db-88ac-efde98cb2539 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:11:25.757814 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fdb5bf6-bc1a-49db-88ac-efde98cb2539 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:11:25.769768 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application [None req-9fdb5bf6-bc1a-49db-88ac-efde98cb2539 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 15 12:11:25.769768 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:11:25.769768 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:11:25.769768 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:11:25.769768 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:11:25.769768 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:11:25.769768 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:11:25.769768 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:11:25.769768 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:11:25.769768 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:11:25.769768 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:11:25.769768 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:11:25.769768 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 15 12:11:25.769768 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 15 12:11:25.769768 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 15 12:11:25.769768 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 15 12:11:25.771459 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:11:25.771459 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:11:25.771459 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 15 12:11:25.771459 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 12:11:25.771459 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 15 12:11:25.771459 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 12:11:25.771459 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:11:25.771459 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:11:25.771459 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:11:25.771459 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:11:25.771459 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:11:25.771459 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application with Lock( Sep 15 12:11:25.771459 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:11:25.771459 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:11:25.771459 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:11:25.771459 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:11:25.771459 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:11:25.771459 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:11:25.772593 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:11:25.772593 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:11:25.772593 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 15 12:11:25.772593 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 15 12:11:25.772593 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 15 12:11:25.772593 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 15 12:11:25.772593 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 15 12:11:25.772593 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 15 12:11:25.772593 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 15 12:11:25.772593 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 15 12:11:25.779079 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 310/614] 10.222.0.15 () {56 vars in 1096 bytes} [Thu Sep 15 12:11:25 2022] GET /identity/v3/users/swift => generated 83 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:11:25.780247 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a43792c-a4fd-4a31-8b8e-04c49c2872a0 None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:25.933071 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a43792c-a4fd-4a31-8b8e-04c49c2872a0 None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:25.983870 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44640f1-f077-4d08-8ce0-3a0dd9d58040 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:25.984501 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44640f1-f077-4d08-8ce0-3a0dd9d58040 None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:25.984846 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44640f1-f077-4d08-8ce0-3a0dd9d58040 None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:25.985124 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44640f1-f077-4d08-8ce0-3a0dd9d58040 None admin] PATH_INFO: `/v3/users` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:25.985854 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44640f1-f077-4d08-8ce0-3a0dd9d58040 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81449) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:11:25.986159 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44640f1-f077-4d08-8ce0-3a0dd9d58040 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:11:25.987592 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44640f1-f077-4d08-8ce0-3a0dd9d58040 None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:11:26.014359 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 305/615] 10.222.0.15 () {56 vars in 1135 bytes} [Thu Sep 15 12:11:25 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:11:26.019550 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fdb5bf6-bc1a-49db-88ac-efde98cb2539 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:26.133949 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fdb5bf6-bc1a-49db-88ac-efde98cb2539 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:26.135412 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:11:26.183058 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8c89f4-ed0d-444d-b17f-1aa0dad4daaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:26.183710 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8c89f4-ed0d-444d-b17f-1aa0dad4daaf None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:26.183985 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8c89f4-ed0d-444d-b17f-1aa0dad4daaf None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:26.184251 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8c89f4-ed0d-444d-b17f-1aa0dad4daaf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:26.195914 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8c89f4-ed0d-444d-b17f-1aa0dad4daaf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:11:26.201454 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8c89f4-ed0d-444d-b17f-1aa0dad4daaf None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:11:26.216603 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application [None req-4b8c89f4-ed0d-444d-b17f-1aa0dad4daaf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 12:11:26.216603 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:11:26.216603 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:11:26.216603 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:11:26.216603 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:11:26.216603 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:11:26.216603 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:11:26.216603 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:11:26.216603 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:11:26.216603 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:11:26.216603 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:11:26.216603 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:11:26.216603 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 12:11:26.216603 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 12:11:26.216603 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 12:11:26.216603 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 12:11:26.217661 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:11:26.217661 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:11:26.217661 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:11:26.217661 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:11:26.217661 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:11:26.217661 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:11:26.217661 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:11:26.217661 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application with Lock( Sep 15 12:11:26.217661 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:11:26.217661 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:11:26.217661 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:11:26.217661 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:11:26.217661 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:11:26.217661 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:11:26.217661 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:11:26.217661 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:11:26.217661 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 12:11:26.218564 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 12:11:26.218564 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 12:11:26.218564 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 15 12:11:26.221090 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 311/616] 10.222.0.15 () {56 vars in 1108 bytes} [Thu Sep 15 12:11:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:11:26.224232 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44640f1-f077-4d08-8ce0-3a0dd9d58040 None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:26.304261 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:11:26.372268 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44640f1-f077-4d08-8ce0-3a0dd9d58040 None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:26.407902 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598f3797-32f2-409f-9762-be52315d775b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:26.408433 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598f3797-32f2-409f-9762-be52315d775b None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:26.408715 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598f3797-32f2-409f-9762-be52315d775b None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:26.409016 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598f3797-32f2-409f-9762-be52315d775b None admin] PATH_INFO: `/v3/domains` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:26.409658 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598f3797-32f2-409f-9762-be52315d775b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81449) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:11:26.409960 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598f3797-32f2-409f-9762-be52315d775b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:11:26.411909 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598f3797-32f2-409f-9762-be52315d775b None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:11:26.424119 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 306/617] 10.222.0.15 () {56 vars in 1109 bytes} [Thu Sep 15 12:11:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:11:26.429422 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8c89f4-ed0d-444d-b17f-1aa0dad4daaf None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:26.564820 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8c89f4-ed0d-444d-b17f-1aa0dad4daaf None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:26.611471 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9927da-7d5f-460a-b7f6-870ac4a67863 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:26.612126 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9927da-7d5f-460a-b7f6-870ac4a67863 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:26.612403 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9927da-7d5f-460a-b7f6-870ac4a67863 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:26.612674 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9927da-7d5f-460a-b7f6-870ac4a67863 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:26.620331 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9927da-7d5f-460a-b7f6-870ac4a67863 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:11:26.621801 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9927da-7d5f-460a-b7f6-870ac4a67863 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:11:26.633348 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application [None req-7e9927da-7d5f-460a-b7f6-870ac4a67863 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 12:11:26.633348 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:11:26.633348 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:11:26.633348 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:11:26.633348 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:11:26.633348 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:11:26.633348 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:11:26.633348 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:11:26.633348 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:11:26.633348 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:11:26.633348 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:11:26.633348 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:11:26.633348 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 15 12:11:26.633348 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 15 12:11:26.633348 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 15 12:11:26.635529 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 15 12:11:26.635529 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:11:26.635529 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:11:26.635529 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:11:26.635529 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:11:26.635529 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:11:26.635529 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:11:26.635529 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:11:26.635529 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application with Lock( Sep 15 12:11:26.635529 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:11:26.635529 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:11:26.635529 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:11:26.635529 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:11:26.635529 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:11:26.635529 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:11:26.635529 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:11:26.635529 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:11:26.637157 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 15 12:11:26.637157 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 15 12:11:26.637157 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 15 12:11:26.637157 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 15 12:11:26.637157 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 15 12:11:26.637157 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 15 12:11:26.637157 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 12:11:26.637157 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 15 12:11:26.637157 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 312/618] 10.222.0.15 () {56 vars in 1111 bytes} [Thu Sep 15 12:11:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:11:26.644200 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598f3797-32f2-409f-9762-be52315d775b None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:26.784796 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598f3797-32f2-409f-9762-be52315d775b None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:26.786520 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:11:26.835892 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee9da97-c0d8-4cc0-bd0f-38da9aa3faa1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:26.836510 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee9da97-c0d8-4cc0-bd0f-38da9aa3faa1 None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:26.836808 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee9da97-c0d8-4cc0-bd0f-38da9aa3faa1 None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:26.837086 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee9da97-c0d8-4cc0-bd0f-38da9aa3faa1 None admin] PATH_INFO: `/v3/projects` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:26.843426 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee9da97-c0d8-4cc0-bd0f-38da9aa3faa1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81449) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:11:26.843426 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee9da97-c0d8-4cc0-bd0f-38da9aa3faa1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:11:26.843426 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee9da97-c0d8-4cc0-bd0f-38da9aa3faa1 None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:11:26.866188 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 307/619] 10.222.0.15 () {56 vars in 1148 bytes} [Thu Sep 15 12:11:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:11:26.873812 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9927da-7d5f-460a-b7f6-870ac4a67863 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:26.901845 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:11:26.923201 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4ec85a-d9d8-40b0-ac4a-4d52a8e6ddd3 None None] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:26.923482 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4ec85a-d9d8-40b0-ac4a-4d52a8e6ddd3 None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:26.924284 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4ec85a-d9d8-40b0-ac4a-4d52a8e6ddd3 None None] PATH_INFO: `/` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:26.925383 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 308/620] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 12:11:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:11:26.934932 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27c1e7d-3902-4335-86e4-aa1905727c83 None None] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:26.935223 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27c1e7d-3902-4335-86e4-aa1905727c83 None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:26.935502 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27c1e7d-3902-4335-86e4-aa1905727c83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:26.981843 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.auth.core [None req-a27c1e7d-3902-4335-86e4-aa1905727c83 None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81449) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:11:26.983188 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-a27c1e7d-3902-4335-86e4-aa1905727c83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:11:27.025315 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9927da-7d5f-460a-b7f6-870ac4a67863 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:27.092533 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97876319-60cc-4703-9391-366a2e2d71bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:27.093155 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97876319-60cc-4703-9391-366a2e2d71bd None admin] REQUEST_METHOD: `PUT` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:27.093441 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97876319-60cc-4703-9391-366a2e2d71bd None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:27.093714 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97876319-60cc-4703-9391-366a2e2d71bd None admin] PATH_INFO: `/v3/projects/540a32ef88764e8c8d9201ae22f6ddaf/users/efa0d3dc5b7540d59fdd59bbf64a04b6/roles/e1cecc29e028497f9cdaa04268c6a6ec` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:27.102032 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 309/621] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 12:11:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 12:11:27.123815 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97876319-60cc-4703-9391-366a2e2d71bd None admin] RBAC: Authorizing `identity:create_grant(project_id=540a32ef88764e8c8d9201ae22f6ddaf, user_id=efa0d3dc5b7540d59fdd59bbf64a04b6, role_id=e1cecc29e028497f9cdaa04268c6a6ec)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:11:27.125396 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97876319-60cc-4703-9391-366a2e2d71bd None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:11:27.152252 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 313/622] 10.222.0.15 () {58 vars in 1439 bytes} [Thu Sep 15 12:11:26 2022] PUT /identity/v3/projects/540a32ef88764e8c8d9201ae22f6ddaf/users/efa0d3dc5b7540d59fdd59bbf64a04b6/roles/e1cecc29e028497f9cdaa04268c6a6ec => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:11:27.448540 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27c1e7d-3902-4335-86e4-aa1905727c83 None None] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:27.555880 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:11:27.568605 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27c1e7d-3902-4335-86e4-aa1905727c83 None None] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:27.619089 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76244498-f88f-4125-a47e-9b6b13c4250c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:27.619761 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76244498-f88f-4125-a47e-9b6b13c4250c None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:27.620042 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76244498-f88f-4125-a47e-9b6b13c4250c None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:27.620315 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76244498-f88f-4125-a47e-9b6b13c4250c None admin] PATH_INFO: `/v3/roles/service` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:27.638271 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76244498-f88f-4125-a47e-9b6b13c4250c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 12:11:27.643144 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application [None req-76244498-f88f-4125-a47e-9b6b13c4250c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 15 12:11:27.643144 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:11:27.643144 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:11:27.643144 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:11:27.643144 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:11:27.643144 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:11:27.643144 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:11:27.643144 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:11:27.643144 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:11:27.643144 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:11:27.643144 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:11:27.643144 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:11:27.643144 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 12:11:27.643144 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 12:11:27.643144 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 12:11:27.643144 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 12:11:27.644143 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 12:11:27.644143 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 12:11:27.644143 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 12:11:27.644143 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 12:11:27.644143 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:11:27.644143 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:11:27.644143 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:11:27.644143 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:11:27.644143 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:11:27.644143 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:11:27.644143 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:11:27.644143 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application with Lock( Sep 15 12:11:27.644143 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:11:27.644143 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:11:27.644143 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:11:27.644143 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:11:27.645064 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:11:27.645064 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:11:27.645064 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:11:27.645064 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:11:27.645064 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 12:11:27.645064 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 12:11:27.645064 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 12:11:27.645064 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 12:11:27.645064 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 12:11:27.645064 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 12:11:27.645064 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 15 12:11:27.645064 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application Sep 15 12:11:27.645064 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 310/623] 10.222.0.15 () {56 vars in 1102 bytes} [Thu Sep 15 12:11:27 2022] GET /identity/v3/roles/service => generated 85 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:11:27.650827 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97876319-60cc-4703-9391-366a2e2d71bd None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:27.771122 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27808b4d-a3f3-4c26-bc01-ed80c3ebc157 None None] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:27.771473 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27808b4d-a3f3-4c26-bc01-ed80c3ebc157 None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:27.771929 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27808b4d-a3f3-4c26-bc01-ed80c3ebc157 None None] PATH_INFO: `/` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:27.777407 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 311/624] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 12:11:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:11:27.779473 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65961920-3a58-4ae5-b6f7-501784956665 None None] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:27.779757 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65961920-3a58-4ae5-b6f7-501784956665 None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:27.780034 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65961920-3a58-4ae5-b6f7-501784956665 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:27.791457 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:11:27.818297 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.auth.core [None req-65961920-3a58-4ae5-b6f7-501784956665 None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81449) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:11:27.819507 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-65961920-3a58-4ae5-b6f7-501784956665 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:11:27.895234 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97876319-60cc-4703-9391-366a2e2d71bd None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:28.028513 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cbc4292-d2b4-4e5d-8d1d-f10d10e33a71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:28.037515 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbc4292-d2b4-4e5d-8d1d-f10d10e33a71 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:28.037515 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbc4292-d2b4-4e5d-8d1d-f10d10e33a71 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:28.037515 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbc4292-d2b4-4e5d-8d1d-f10d10e33a71 None admin] PATH_INFO: `/v3/roles` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:28.037515 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cbc4292-d2b4-4e5d-8d1d-f10d10e33a71 None admin] RBAC: Adding query filter params (name=service) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:11:28.037515 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cbc4292-d2b4-4e5d-8d1d-f10d10e33a71 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:11:28.037515 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cbc4292-d2b4-4e5d-8d1d-f10d10e33a71 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:11:28.044957 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 312/625] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 12:11:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 266 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 12:11:28.070991 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 314/626] 10.222.0.15 () {56 vars in 1103 bytes} [Thu Sep 15 12:11:27 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 421 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:11:28.077075 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65961920-3a58-4ae5-b6f7-501784956665 None None] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:28.173667 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0469dce2-a2b0-4f4e-97e7-d34f31fab008 None None] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:28.174223 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0469dce2-a2b0-4f4e-97e7-d34f31fab008 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:28.174463 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0469dce2-a2b0-4f4e-97e7-d34f31fab008 None None] PATH_INFO: `/` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:28.177345 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 315/627] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 12:11:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 15 12:11:28.187229 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a03ab55-cad3-4e2a-8508-6e0af59ca7ac None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:28.187229 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a03ab55-cad3-4e2a-8508-6e0af59ca7ac None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:28.187229 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a03ab55-cad3-4e2a-8508-6e0af59ca7ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:28.257429 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-6a03ab55-cad3-4e2a-8508-6e0af59ca7ac None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:11:28.258626 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-6a03ab55-cad3-4e2a-8508-6e0af59ca7ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:11:28.285647 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:11:28.316333 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65961920-3a58-4ae5-b6f7-501784956665 None None] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:28.360685 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855d71ca-887d-427b-b656-033fac160a5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:28.361929 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855d71ca-887d-427b-b656-033fac160a5c None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:28.362415 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855d71ca-887d-427b-b656-033fac160a5c None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:28.362901 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-855d71ca-887d-427b-b656-033fac160a5c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:28.370274 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 316/628] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 12:11:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 12:11:28.371916 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4bca14-b216-476e-84a0-0c989afc5be3 None None] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:28.372042 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4bca14-b216-476e-84a0-0c989afc5be3 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:28.372244 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef4bca14-b216-476e-84a0-0c989afc5be3 None None] PATH_INFO: `/` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:28.373780 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 317/629] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 12:11:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:11:28.376152 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855d71ca-887d-427b-b656-033fac160a5c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:11:28.380840 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-855d71ca-887d-427b-b656-033fac160a5c None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:11:28.383843 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f07c0a-0f0f-4e3f-9b93-d4d43948905d None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:28.383843 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f07c0a-0f0f-4e3f-9b93-d4d43948905d None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:28.383843 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f07c0a-0f0f-4e3f-9b93-d4d43948905d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:28.394164 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application [None req-855d71ca-887d-427b-b656-033fac160a5c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 12:11:28.394164 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:11:28.394164 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:11:28.394164 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:11:28.394164 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:11:28.394164 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:11:28.394164 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:11:28.394164 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:11:28.394164 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:11:28.394164 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:11:28.394164 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:11:28.394164 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:11:28.394164 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 12:11:28.394164 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 12:11:28.394164 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 12:11:28.394164 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 12:11:28.395141 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:11:28.395141 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:11:28.395141 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:11:28.395141 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:11:28.395141 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:11:28.395141 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:11:28.395141 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:11:28.395141 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application with Lock( Sep 15 12:11:28.395141 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:11:28.395141 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:11:28.395141 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:11:28.395141 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:11:28.395141 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:11:28.395141 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:11:28.395141 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:11:28.395141 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:11:28.395141 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 12:11:28.396381 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 12:11:28.396381 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 12:11:28.396381 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application Sep 15 12:11:28.396381 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 313/630] 10.222.0.15 () {56 vars in 1108 bytes} [Thu Sep 15 12:11:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 319 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:11:28.401374 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855d71ca-887d-427b-b656-033fac160a5c None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:28.412501 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-14f07c0a-0f0f-4e3f-9b93-d4d43948905d None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:11:28.413750 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-14f07c0a-0f0f-4e3f-9b93-d4d43948905d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:11:28.471036 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:11:28.491622 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 318/631] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 12:11:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 12:11:28.498279 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f07c0a-0f0f-4e3f-9b93-d4d43948905d None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:28.522756 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-855d71ca-887d-427b-b656-033fac160a5c None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:28.562584 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe58f26-4c6e-4d4b-a7e1-e7906fb07e68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:28.563178 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe58f26-4c6e-4d4b-a7e1-e7906fb07e68 None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:28.563459 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe58f26-4c6e-4d4b-a7e1-e7906fb07e68 None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:28.563729 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe58f26-4c6e-4d4b-a7e1-e7906fb07e68 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:28.574043 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe58f26-4c6e-4d4b-a7e1-e7906fb07e68 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 12:11:28.579794 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application [None req-5fe58f26-4c6e-4d4b-a7e1-e7906fb07e68 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 15 12:11:28.579794 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:11:28.579794 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:11:28.579794 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:11:28.579794 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:11:28.579794 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:11:28.579794 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:11:28.579794 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:11:28.579794 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:11:28.579794 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:11:28.579794 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:11:28.579794 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:11:28.579794 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 12:11:28.579794 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 12:11:28.579794 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 12:11:28.579794 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 12:11:28.580768 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 12:11:28.580768 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 12:11:28.580768 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 12:11:28.580768 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 12:11:28.580768 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:11:28.580768 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:11:28.580768 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:11:28.580768 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:11:28.580768 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:11:28.580768 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:11:28.580768 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:11:28.580768 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application with Lock( Sep 15 12:11:28.580768 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:11:28.580768 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:11:28.580768 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:11:28.580768 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:11:28.581649 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:11:28.581649 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:11:28.581649 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:11:28.581649 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:11:28.581649 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 12:11:28.581649 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 12:11:28.581649 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 12:11:28.581649 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 12:11:28.581649 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 12:11:28.581649 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 12:11:28.581649 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 15 12:11:28.581649 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application Sep 15 12:11:28.581649 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:11:28.585484 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 314/632] 10.222.0.15 () {56 vars in 1102 bytes} [Thu Sep 15 12:11:28 2022] GET /identity/v3/roles/service => generated 85 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 15 12:11:28.587601 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe58f26-4c6e-4d4b-a7e1-e7906fb07e68 None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:28.674860 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f07c0a-0f0f-4e3f-9b93-d4d43948905d None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:28.706882 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe58f26-4c6e-4d4b-a7e1-e7906fb07e68 None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:28.747255 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98fe581d-dba3-42a8-b0f4-8c69fb3ec2cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:28.747912 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fe581d-dba3-42a8-b0f4-8c69fb3ec2cb None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:28.748289 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fe581d-dba3-42a8-b0f4-8c69fb3ec2cb None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:28.748660 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fe581d-dba3-42a8-b0f4-8c69fb3ec2cb None admin] PATH_INFO: `/v3/roles` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:28.749357 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98fe581d-dba3-42a8-b0f4-8c69fb3ec2cb None admin] RBAC: Adding query filter params (name=service) {{(pid=81449) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:11:28.749561 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25ff82b-8cb2-4f48-aa33-99a0af19d1c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:28.749690 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98fe581d-dba3-42a8-b0f4-8c69fb3ec2cb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:11:28.750181 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25ff82b-8cb2-4f48-aa33-99a0af19d1c2 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:28.750460 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25ff82b-8cb2-4f48-aa33-99a0af19d1c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:28.750737 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25ff82b-8cb2-4f48-aa33-99a0af19d1c2 None admin] PATH_INFO: `/v3/domains` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:28.750994 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98fe581d-dba3-42a8-b0f4-8c69fb3ec2cb None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:11:28.751379 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25ff82b-8cb2-4f48-aa33-99a0af19d1c2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:11:28.751683 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25ff82b-8cb2-4f48-aa33-99a0af19d1c2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:11:28.753241 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25ff82b-8cb2-4f48-aa33-99a0af19d1c2 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:11:28.760628 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 315/633] 10.222.0.15 () {56 vars in 1103 bytes} [Thu Sep 15 12:11:28 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:11:28.766190 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98fe581d-dba3-42a8-b0f4-8c69fb3ec2cb None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:28.766349 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 319/634] 10.222.0.15 () {56 vars in 1109 bytes} [Thu Sep 15 12:11:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:11:28.767380 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25ff82b-8cb2-4f48-aa33-99a0af19d1c2 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:28.823115 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:11:28.825727 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:11:28.871426 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98fe581d-dba3-42a8-b0f4-8c69fb3ec2cb None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:28.881085 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25ff82b-8cb2-4f48-aa33-99a0af19d1c2 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:28.906119 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d598068e-1385-43a1-86ed-77ef35aa50ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:28.906745 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d598068e-1385-43a1-86ed-77ef35aa50ab None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:28.907025 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d598068e-1385-43a1-86ed-77ef35aa50ab None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:28.907297 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d598068e-1385-43a1-86ed-77ef35aa50ab None admin] PATH_INFO: `/v3/roles/service` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:28.916994 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113b9eec-c3e1-42a7-9713-4f5e2cbb35e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:28.917986 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d598068e-1385-43a1-86ed-77ef35aa50ab None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 12:11:28.918248 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113b9eec-c3e1-42a7-9713-4f5e2cbb35e4 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:28.918673 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113b9eec-c3e1-42a7-9713-4f5e2cbb35e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:28.919098 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113b9eec-c3e1-42a7-9713-4f5e2cbb35e4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:28.923100 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application [None req-d598068e-1385-43a1-86ed-77ef35aa50ab None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 15 12:11:28.923100 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:11:28.923100 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:11:28.923100 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:11:28.923100 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:11:28.923100 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:11:28.923100 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:11:28.923100 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:11:28.923100 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:11:28.923100 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:11:28.923100 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:11:28.923100 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:11:28.923100 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 12:11:28.923100 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 12:11:28.923100 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 12:11:28.923100 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 12:11:28.924227 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 12:11:28.924227 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 12:11:28.924227 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 12:11:28.924227 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 12:11:28.924227 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:11:28.924227 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:11:28.924227 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:11:28.924227 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:11:28.924227 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:11:28.924227 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:11:28.924227 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:11:28.924227 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application with Lock( Sep 15 12:11:28.924227 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:11:28.924227 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:11:28.924227 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:11:28.924227 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:11:28.925328 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:11:28.925328 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:11:28.925328 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:11:28.925328 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:11:28.925328 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 12:11:28.925328 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 12:11:28.925328 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 12:11:28.925328 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 12:11:28.925328 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 12:11:28.925328 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 12:11:28.925328 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 15 12:11:28.925328 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application Sep 15 12:11:28.925328 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 316/635] 10.222.0.15 () {56 vars in 1102 bytes} [Thu Sep 15 12:11:28 2022] GET /identity/v3/roles/service => generated 85 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:11:28.925328 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d598068e-1385-43a1-86ed-77ef35aa50ab None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:28.934546 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-113b9eec-c3e1-42a7-9713-4f5e2cbb35e4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:11:28.936156 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-113b9eec-c3e1-42a7-9713-4f5e2cbb35e4 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:11:28.951664 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application [None req-113b9eec-c3e1-42a7-9713-4f5e2cbb35e4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 12:11:28.951664 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:11:28.951664 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:11:28.951664 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:11:28.951664 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:11:28.951664 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:11:28.951664 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:11:28.951664 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:11:28.951664 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:11:28.951664 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:11:28.951664 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:11:28.951664 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:11:28.951664 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 12:11:28.951664 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 12:11:28.951664 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 12:11:28.951664 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 12:11:28.952830 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:11:28.952830 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:11:28.952830 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:11:28.952830 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:11:28.952830 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:11:28.952830 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:11:28.952830 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:11:28.952830 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application with Lock( Sep 15 12:11:28.952830 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:11:28.952830 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:11:28.952830 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:11:28.952830 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:11:28.952830 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:11:28.952830 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:11:28.952830 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:11:28.952830 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:11:28.952830 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 12:11:28.953940 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 12:11:28.953940 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 12:11:28.953940 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 15 12:11:28.960615 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 320/636] 10.222.0.15 () {56 vars in 1108 bytes} [Thu Sep 15 12:11:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:11:28.961716 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113b9eec-c3e1-42a7-9713-4f5e2cbb35e4 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:29.038545 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d598068e-1385-43a1-86ed-77ef35aa50ab None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:29.055518 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:11:29.062750 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:11:29.066920 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113b9eec-c3e1-42a7-9713-4f5e2cbb35e4 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:29.080095 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8e75a8-bcb3-4999-a932-feab4b8796a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:29.080686 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8e75a8-bcb3-4999-a932-feab4b8796a9 None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:29.081014 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8e75a8-bcb3-4999-a932-feab4b8796a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:29.081298 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8e75a8-bcb3-4999-a932-feab4b8796a9 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:29.089120 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8e75a8-bcb3-4999-a932-feab4b8796a9 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:11:29.090641 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8e75a8-bcb3-4999-a932-feab4b8796a9 None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:11:29.102658 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5c57a9-097a-45fa-b07d-f19599fc0723 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:29.102824 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application [None req-6b8e75a8-bcb3-4999-a932-feab4b8796a9 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 15 12:11:29.102824 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:11:29.102824 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:11:29.102824 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:11:29.102824 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:11:29.102824 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:11:29.102824 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:11:29.102824 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:11:29.102824 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:11:29.102824 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:11:29.102824 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:11:29.102824 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:11:29.102824 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 15 12:11:29.102824 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 15 12:11:29.102824 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 15 12:11:29.102824 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 15 12:11:29.103716 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:11:29.103716 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:11:29.103716 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 15 12:11:29.103716 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 12:11:29.103716 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 15 12:11:29.103716 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 12:11:29.103716 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:11:29.103716 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:11:29.103716 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:11:29.103716 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:11:29.103716 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:11:29.103716 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application with Lock( Sep 15 12:11:29.103716 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:11:29.103716 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:11:29.103716 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:11:29.103716 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:11:29.103716 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:11:29.103716 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:11:29.105000 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:11:29.105000 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:11:29.105000 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 15 12:11:29.105000 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 15 12:11:29.105000 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 15 12:11:29.105000 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 15 12:11:29.105000 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 15 12:11:29.105000 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 15 12:11:29.105000 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 15 12:11:29.105000 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application Sep 15 12:11:29.105550 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5c57a9-097a-45fa-b07d-f19599fc0723 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:29.105550 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5c57a9-097a-45fa-b07d-f19599fc0723 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:29.105550 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5c57a9-097a-45fa-b07d-f19599fc0723 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:29.105736 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 317/637] 10.222.0.15 () {56 vars in 1093 bytes} [Thu Sep 15 12:11:28 2022] GET /identity/v3/users/nova => generated 82 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:11:29.105736 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8e75a8-bcb3-4999-a932-feab4b8796a9 None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:29.119975 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af5c57a9-097a-45fa-b07d-f19599fc0723 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 12:11:29.125238 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application [None req-af5c57a9-097a-45fa-b07d-f19599fc0723 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 15 12:11:29.125238 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:11:29.125238 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:11:29.125238 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:11:29.125238 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:11:29.125238 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:11:29.125238 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:11:29.125238 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:11:29.125238 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:11:29.125238 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:11:29.125238 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:11:29.125238 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:11:29.125238 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 12:11:29.125238 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 12:11:29.125238 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 12:11:29.125238 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 12:11:29.126158 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 12:11:29.126158 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 12:11:29.126158 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 12:11:29.126158 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 12:11:29.126158 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:11:29.126158 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:11:29.126158 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:11:29.126158 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:11:29.126158 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:11:29.126158 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:11:29.126158 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:11:29.126158 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application with Lock( Sep 15 12:11:29.126158 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:11:29.126158 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:11:29.126158 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:11:29.126158 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:11:29.127075 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:11:29.127075 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:11:29.127075 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:11:29.127075 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:11:29.127075 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 12:11:29.127075 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 12:11:29.127075 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 12:11:29.127075 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 12:11:29.127075 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 12:11:29.127075 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 12:11:29.127075 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 15 12:11:29.127075 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 15 12:11:29.127075 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 321/638] 10.222.0.15 () {56 vars in 1102 bytes} [Thu Sep 15 12:11:28 2022] GET /identity/v3/roles/service => generated 85 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:11:29.127075 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5c57a9-097a-45fa-b07d-f19599fc0723 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:29.226307 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8e75a8-bcb3-4999-a932-feab4b8796a9 None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:29.240476 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5c57a9-097a-45fa-b07d-f19599fc0723 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:29.263939 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17428b5-f1c3-423e-a9e9-167e8df38a94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:29.264550 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17428b5-f1c3-423e-a9e9-167e8df38a94 None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:29.264860 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17428b5-f1c3-423e-a9e9-167e8df38a94 None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:29.265172 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17428b5-f1c3-423e-a9e9-167e8df38a94 None admin] PATH_INFO: `/v3/roles` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:29.266043 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17428b5-f1c3-423e-a9e9-167e8df38a94 None admin] RBAC: Adding query filter params (name=service) {{(pid=81449) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:11:29.266151 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17428b5-f1c3-423e-a9e9-167e8df38a94 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:11:29.267424 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17428b5-f1c3-423e-a9e9-167e8df38a94 None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:11:29.279849 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 318/639] 10.222.0.15 () {56 vars in 1103 bytes} [Thu Sep 15 12:11:29 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:11:29.279849 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17428b5-f1c3-423e-a9e9-167e8df38a94 None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:29.282445 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9227027a-593f-45b6-9cac-98f1c6577287 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:29.283065 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9227027a-593f-45b6-9cac-98f1c6577287 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:29.283347 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9227027a-593f-45b6-9cac-98f1c6577287 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:29.283624 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9227027a-593f-45b6-9cac-98f1c6577287 None admin] PATH_INFO: `/v3/domains` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:29.284269 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9227027a-593f-45b6-9cac-98f1c6577287 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:11:29.284572 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9227027a-593f-45b6-9cac-98f1c6577287 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:11:29.285953 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9227027a-593f-45b6-9cac-98f1c6577287 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:11:29.298924 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 322/640] 10.222.0.15 () {56 vars in 1109 bytes} [Thu Sep 15 12:11:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:11:29.299737 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9227027a-593f-45b6-9cac-98f1c6577287 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:29.320572 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:11:29.330788 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:11:29.393134 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17428b5-f1c3-423e-a9e9-167e8df38a94 None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:29.408953 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9227027a-593f-45b6-9cac-98f1c6577287 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:29.430228 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1587a5a-39e1-4173-8436-0f18f6f8060b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:29.430841 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1587a5a-39e1-4173-8436-0f18f6f8060b None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:29.431122 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1587a5a-39e1-4173-8436-0f18f6f8060b None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:29.431391 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1587a5a-39e1-4173-8436-0f18f6f8060b None admin] PATH_INFO: `/v3/users` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:29.432219 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1587a5a-39e1-4173-8436-0f18f6f8060b None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81449) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:11:29.432527 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1587a5a-39e1-4173-8436-0f18f6f8060b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:11:29.434109 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1587a5a-39e1-4173-8436-0f18f6f8060b None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:11:29.443683 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8948b4-7473-4917-b591-6d7c384248a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:29.444335 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8948b4-7473-4917-b591-6d7c384248a1 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:29.444613 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8948b4-7473-4917-b591-6d7c384248a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:29.444912 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8948b4-7473-4917-b591-6d7c384248a1 None admin] PATH_INFO: `/v3/roles` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:29.445559 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a8948b4-7473-4917-b591-6d7c384248a1 None admin] RBAC: Adding query filter params (name=service) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:11:29.445854 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a8948b4-7473-4917-b591-6d7c384248a1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:11:29.447211 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a8948b4-7473-4917-b591-6d7c384248a1 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:11:29.453129 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 319/641] 10.222.0.15 () {56 vars in 1133 bytes} [Thu Sep 15 12:11:29 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:11:29.454297 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1587a5a-39e1-4173-8436-0f18f6f8060b None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:29.460347 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 323/642] 10.222.0.15 () {56 vars in 1103 bytes} [Thu Sep 15 12:11:29 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:11:29.461339 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8948b4-7473-4917-b591-6d7c384248a1 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:29.550302 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:11:29.551975 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:11:29.559586 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1587a5a-39e1-4173-8436-0f18f6f8060b None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:29.571438 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8948b4-7473-4917-b591-6d7c384248a1 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:29.596722 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d30ba1-8a31-411f-a414-892d44a484b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:29.597311 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d30ba1-8a31-411f-a414-892d44a484b6 None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:29.597589 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d30ba1-8a31-411f-a414-892d44a484b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:29.597861 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d30ba1-8a31-411f-a414-892d44a484b6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:29.607850 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99eeb482-92a7-4158-88e9-0bee9cabbd94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:29.608484 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99eeb482-92a7-4158-88e9-0bee9cabbd94 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:29.608776 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99eeb482-92a7-4158-88e9-0bee9cabbd94 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:29.609057 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99eeb482-92a7-4158-88e9-0bee9cabbd94 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:29.609158 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d30ba1-8a31-411f-a414-892d44a484b6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:11:29.610491 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d30ba1-8a31-411f-a414-892d44a484b6 None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:11:29.617229 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99eeb482-92a7-4158-88e9-0bee9cabbd94 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:11:29.618711 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99eeb482-92a7-4158-88e9-0bee9cabbd94 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:11:29.626383 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application [None req-86d30ba1-8a31-411f-a414-892d44a484b6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 12:11:29.626383 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:11:29.626383 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:11:29.626383 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:11:29.626383 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:11:29.626383 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:11:29.626383 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:11:29.626383 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:11:29.626383 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:11:29.626383 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:11:29.626383 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:11:29.626383 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:11:29.626383 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 12:11:29.626383 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 12:11:29.626383 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 12:11:29.626383 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 12:11:29.627483 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:11:29.627483 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:11:29.627483 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:11:29.627483 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:11:29.627483 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:11:29.627483 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:11:29.627483 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:11:29.627483 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application with Lock( Sep 15 12:11:29.627483 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:11:29.627483 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:11:29.627483 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:11:29.627483 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:11:29.627483 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:11:29.627483 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:11:29.627483 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:11:29.627483 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:11:29.627483 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 12:11:29.628418 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 12:11:29.628418 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 12:11:29.628418 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application Sep 15 12:11:29.631328 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application [None req-99eeb482-92a7-4158-88e9-0bee9cabbd94 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 15 12:11:29.631328 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:11:29.631328 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:11:29.631328 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:11:29.631328 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:11:29.631328 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:11:29.631328 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:11:29.631328 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:11:29.631328 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:11:29.631328 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:11:29.631328 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:11:29.631328 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:11:29.631328 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 15 12:11:29.631328 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 15 12:11:29.631328 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 15 12:11:29.631328 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 15 12:11:29.632885 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:11:29.632885 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:11:29.632885 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 15 12:11:29.632885 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 12:11:29.632885 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 15 12:11:29.632885 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 12:11:29.632885 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:11:29.632885 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:11:29.632885 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:11:29.632885 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:11:29.632885 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:11:29.632885 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application with Lock( Sep 15 12:11:29.632885 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:11:29.632885 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:11:29.632885 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:11:29.632885 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:11:29.632885 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:11:29.632885 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:11:29.634600 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:11:29.634600 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:11:29.634600 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 15 12:11:29.634600 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 15 12:11:29.634600 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 15 12:11:29.634600 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 15 12:11:29.634600 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 15 12:11:29.634600 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 15 12:11:29.634600 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 15 12:11:29.634600 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 15 12:11:29.635892 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 320/643] 10.222.0.15 () {56 vars in 1108 bytes} [Thu Sep 15 12:11:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:11:29.636055 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 324/644] 10.222.0.15 () {56 vars in 1099 bytes} [Thu Sep 15 12:11:29 2022] GET /identity/v3/users/cinder => generated 84 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:11:29.636207 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d30ba1-8a31-411f-a414-892d44a484b6 None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:29.636354 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99eeb482-92a7-4158-88e9-0bee9cabbd94 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:29.740059 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d30ba1-8a31-411f-a414-892d44a484b6 None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:29.746824 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99eeb482-92a7-4158-88e9-0bee9cabbd94 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:29.774652 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f381d8c1-ded2-4707-8e44-2c8f16576f70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:29.775255 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f381d8c1-ded2-4707-8e44-2c8f16576f70 None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:29.775529 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f381d8c1-ded2-4707-8e44-2c8f16576f70 None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:29.775797 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f381d8c1-ded2-4707-8e44-2c8f16576f70 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:29.782492 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05389a85-9788-4c68-9c89-e40e20a66a47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:29.783126 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05389a85-9788-4c68-9c89-e40e20a66a47 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:29.783405 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05389a85-9788-4c68-9c89-e40e20a66a47 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:29.783674 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05389a85-9788-4c68-9c89-e40e20a66a47 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:29.792283 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:11:29.792607 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:11:29.794750 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05389a85-9788-4c68-9c89-e40e20a66a47 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:11:29.796836 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05389a85-9788-4c68-9c89-e40e20a66a47 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:11:29.811155 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application [None req-05389a85-9788-4c68-9c89-e40e20a66a47 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 12:11:29.811155 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:11:29.811155 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:11:29.811155 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:11:29.811155 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:11:29.811155 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:11:29.811155 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:11:29.811155 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:11:29.811155 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:11:29.811155 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:11:29.811155 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:11:29.811155 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:11:29.811155 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 12:11:29.811155 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 12:11:29.811155 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 12:11:29.811155 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 12:11:29.812288 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:11:29.812288 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:11:29.812288 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:11:29.812288 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:11:29.812288 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:11:29.812288 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:11:29.812288 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:11:29.812288 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application with Lock( Sep 15 12:11:29.812288 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:11:29.812288 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:11:29.812288 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:11:29.812288 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:11:29.812288 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:11:29.812288 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:11:29.812288 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:11:29.812288 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:11:29.812288 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 12:11:29.813222 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 12:11:29.813222 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 12:11:29.813222 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 15 12:11:29.813222 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 325/645] 10.222.0.15 () {56 vars in 1108 bytes} [Thu Sep 15 12:11:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:11:29.813458 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05389a85-9788-4c68-9c89-e40e20a66a47 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:29.845302 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f381d8c1-ded2-4707-8e44-2c8f16576f70 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:11:29.846797 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f381d8c1-ded2-4707-8e44-2c8f16576f70 None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:11:29.912371 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05389a85-9788-4c68-9c89-e40e20a66a47 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:29.940912 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 321/646] 10.222.0.15 () {58 vars in 1282 bytes} [Thu Sep 15 12:11:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 15 12:11:29.941840 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f381d8c1-ded2-4707-8e44-2c8f16576f70 None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:29.945709 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bbe4c71-f515-42f2-bba8-dc29a93f137e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:29.946702 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbe4c71-f515-42f2-bba8-dc29a93f137e None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:29.947197 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbe4c71-f515-42f2-bba8-dc29a93f137e None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:29.947686 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbe4c71-f515-42f2-bba8-dc29a93f137e None admin] PATH_INFO: `/v3/domains` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:29.948492 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bbe4c71-f515-42f2-bba8-dc29a93f137e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:11:29.948823 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bbe4c71-f515-42f2-bba8-dc29a93f137e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:11:29.951523 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bbe4c71-f515-42f2-bba8-dc29a93f137e None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:11:29.971100 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 326/647] 10.222.0.15 () {56 vars in 1109 bytes} [Thu Sep 15 12:11:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:11:29.972899 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a0983e-c7be-4df4-a598-57926f4618e5 None None] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:29.973187 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a0983e-c7be-4df4-a598-57926f4618e5 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:29.973459 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a0983e-c7be-4df4-a598-57926f4618e5 None None] PATH_INFO: `/` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:29.974676 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 327/648] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 12:11:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:11:29.975457 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a0983e-c7be-4df4-a598-57926f4618e5 None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:29.998686 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:11:30.030048 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:11:30.049025 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f381d8c1-ded2-4707-8e44-2c8f16576f70 None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:30.077142 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a0983e-c7be-4df4-a598-57926f4618e5 None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:30.081818 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7bc8e0c-4290-449b-8340-2b2be500793e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:30.082419 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bc8e0c-4290-449b-8340-2b2be500793e None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:30.082697 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bc8e0c-4290-449b-8340-2b2be500793e None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:30.083597 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bc8e0c-4290-449b-8340-2b2be500793e None admin] PATH_INFO: `/v3/users` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:30.084314 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7bc8e0c-4290-449b-8340-2b2be500793e None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81449) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:11:30.084615 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7bc8e0c-4290-449b-8340-2b2be500793e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:11:30.086060 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7bc8e0c-4290-449b-8340-2b2be500793e None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:11:30.103448 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 322/649] 10.222.0.15 () {56 vars in 1137 bytes} [Thu Sep 15 12:11:29 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:11:30.104175 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7bc8e0c-4290-449b-8340-2b2be500793e None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:30.112278 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4226dc94-2a70-4891-b821-890557e21187 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:30.113002 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4226dc94-2a70-4891-b821-890557e21187 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:30.113285 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4226dc94-2a70-4891-b821-890557e21187 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:30.113556 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4226dc94-2a70-4891-b821-890557e21187 None admin] PATH_INFO: `/v3/domains` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:30.114239 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4226dc94-2a70-4891-b821-890557e21187 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:11:30.114541 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4226dc94-2a70-4891-b821-890557e21187 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:11:30.115891 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4226dc94-2a70-4891-b821-890557e21187 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:11:30.127454 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 328/650] 10.222.0.15 () {56 vars in 1109 bytes} [Thu Sep 15 12:11:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:11:30.128633 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4226dc94-2a70-4891-b821-890557e21187 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:30.203345 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7bc8e0c-4290-449b-8340-2b2be500793e None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:30.212017 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:11:30.229095 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4226dc94-2a70-4891-b821-890557e21187 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:30.240938 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b0b88a5-167f-4a25-a3a9-a96ca8c8c38e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:30.241564 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0b88a5-167f-4a25-a3a9-a96ca8c8c38e None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:30.241844 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0b88a5-167f-4a25-a3a9-a96ca8c8c38e None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:30.242117 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0b88a5-167f-4a25-a3a9-a96ca8c8c38e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:30.242234 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:11:30.252138 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0b88a5-167f-4a25-a3a9-a96ca8c8c38e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:11:30.253587 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b0b88a5-167f-4a25-a3a9-a96ca8c8c38e None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:11:30.263910 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6108df3-439b-4b16-a3f0-0149bf35453e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:30.264536 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6108df3-439b-4b16-a3f0-0149bf35453e None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:30.264838 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6108df3-439b-4b16-a3f0-0149bf35453e None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:30.265102 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6108df3-439b-4b16-a3f0-0149bf35453e None admin] PATH_INFO: `/v3/users/glance` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:30.268339 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application [None req-4b0b88a5-167f-4a25-a3a9-a96ca8c8c38e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 12:11:30.268339 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:11:30.268339 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:11:30.268339 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:11:30.268339 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:11:30.268339 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:11:30.268339 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:11:30.268339 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:11:30.268339 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:11:30.268339 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:11:30.268339 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:11:30.268339 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:11:30.268339 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 12:11:30.268339 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 12:11:30.268339 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 12:11:30.268339 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 12:11:30.269443 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:11:30.269443 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:11:30.269443 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:11:30.269443 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:11:30.269443 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:11:30.269443 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:11:30.269443 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:11:30.269443 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application with Lock( Sep 15 12:11:30.269443 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:11:30.269443 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:11:30.269443 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:11:30.269443 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:11:30.269443 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:11:30.269443 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:11:30.269443 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:11:30.269443 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:11:30.269443 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 12:11:30.270392 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 12:11:30.270392 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 12:11:30.270392 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application Sep 15 12:11:30.270392 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 323/651] 10.222.0.15 () {56 vars in 1108 bytes} [Thu Sep 15 12:11:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:11:30.270754 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab33376-c9a8-4175-91aa-e022b8dc933d None None] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:30.270930 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab33376-c9a8-4175-91aa-e022b8dc933d None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:30.271206 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab33376-c9a8-4175-91aa-e022b8dc933d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:30.272469 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6108df3-439b-4b16-a3f0-0149bf35453e None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:11:30.274034 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6108df3-439b-4b16-a3f0-0149bf35453e None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:11:30.285052 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application [None req-c6108df3-439b-4b16-a3f0-0149bf35453e None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Sep 15 12:11:30.285052 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:11:30.285052 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:11:30.285052 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:11:30.285052 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:11:30.285052 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:11:30.285052 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:11:30.285052 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:11:30.285052 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:11:30.285052 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:11:30.285052 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:11:30.285052 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:11:30.285052 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 15 12:11:30.285052 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 15 12:11:30.285052 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 15 12:11:30.285052 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 15 12:11:30.286022 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:11:30.286022 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:11:30.286022 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 15 12:11:30.286022 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 12:11:30.286022 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 15 12:11:30.286022 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 12:11:30.286022 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:11:30.286022 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:11:30.286022 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:11:30.286022 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:11:30.286022 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:11:30.286022 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application with Lock( Sep 15 12:11:30.286022 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:11:30.286022 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:11:30.286022 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:11:30.286022 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:11:30.286022 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:11:30.286022 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:11:30.286951 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:11:30.286951 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:11:30.286951 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 15 12:11:30.286951 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 15 12:11:30.286951 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 15 12:11:30.286951 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 15 12:11:30.286951 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 15 12:11:30.286951 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 15 12:11:30.286951 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Sep 15 12:11:30.286951 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 15 12:11:30.286951 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 329/652] 10.222.0.15 () {56 vars in 1099 bytes} [Thu Sep 15 12:11:30 2022] GET /identity/v3/users/glance => generated 84 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:11:30.288051 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6108df3-439b-4b16-a3f0-0149bf35453e None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:30.288745 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.auth.core [None req-dab33376-c9a8-4175-91aa-e022b8dc933d None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81449) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:11:30.289962 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-dab33376-c9a8-4175-91aa-e022b8dc933d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:11:30.356357 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 324/653] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 12:11:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 12:11:30.357146 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab33376-c9a8-4175-91aa-e022b8dc933d None None] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:30.386330 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6108df3-439b-4b16-a3f0-0149bf35453e None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:30.424881 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f838b631-7023-43cc-88c3-db48fba38466 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:30.425994 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f838b631-7023-43cc-88c3-db48fba38466 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:30.426555 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f838b631-7023-43cc-88c3-db48fba38466 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:30.427515 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f838b631-7023-43cc-88c3-db48fba38466 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:30.454813 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:11:30.467421 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab33376-c9a8-4175-91aa-e022b8dc933d None None] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:30.470750 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:11:30.498971 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f838b631-7023-43cc-88c3-db48fba38466 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:11:30.500401 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f838b631-7023-43cc-88c3-db48fba38466 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:11:30.503637 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85fde318-54f0-48d4-b000-cf90042a14fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:30.503809 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fde318-54f0-48d4-b000-cf90042a14fe None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:30.504055 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fde318-54f0-48d4-b000-cf90042a14fe None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:30.504316 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fde318-54f0-48d4-b000-cf90042a14fe None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:30.512580 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85fde318-54f0-48d4-b000-cf90042a14fe None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:11:30.514358 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85fde318-54f0-48d4-b000-cf90042a14fe None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:11:30.528335 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application [None req-85fde318-54f0-48d4-b000-cf90042a14fe None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Sep 15 12:11:30.528335 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:11:30.528335 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:11:30.528335 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:11:30.528335 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:11:30.528335 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:11:30.528335 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:11:30.528335 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:11:30.528335 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:11:30.528335 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:11:30.528335 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:11:30.528335 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:11:30.528335 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 15 12:11:30.528335 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 15 12:11:30.528335 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 15 12:11:30.528335 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 15 12:11:30.529323 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:11:30.529323 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:11:30.529323 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 15 12:11:30.529323 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 12:11:30.529323 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 15 12:11:30.529323 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 12:11:30.529323 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:11:30.529323 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:11:30.529323 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:11:30.529323 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:11:30.529323 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:11:30.529323 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application with Lock( Sep 15 12:11:30.529323 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:11:30.529323 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:11:30.529323 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:11:30.529323 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:11:30.529323 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:11:30.529323 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:11:30.530269 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:11:30.530269 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:11:30.530269 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 15 12:11:30.530269 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 15 12:11:30.530269 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 15 12:11:30.530269 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 15 12:11:30.530269 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 15 12:11:30.530269 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 15 12:11:30.530269 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Sep 15 12:11:30.530269 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application Sep 15 12:11:30.532893 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 325/654] 10.222.0.15 () {56 vars in 1102 bytes} [Thu Sep 15 12:11:30 2022] GET /identity/v3/users/neutron => generated 85 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:11:30.533709 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85fde318-54f0-48d4-b000-cf90042a14fe None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:30.611412 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 330/655] 10.222.0.15 () {58 vars in 1282 bytes} [Thu Sep 15 12:11:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 15 12:11:30.611412 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f838b631-7023-43cc-88c3-db48fba38466 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:30.640264 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85fde318-54f0-48d4-b000-cf90042a14fe None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:30.672969 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f92cc55-d68e-425f-acb4-cd374b407057 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:30.673568 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f92cc55-d68e-425f-acb4-cd374b407057 None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:30.673842 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f92cc55-d68e-425f-acb4-cd374b407057 None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:30.674199 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f92cc55-d68e-425f-acb4-cd374b407057 None admin] PATH_INFO: `/v3/domains` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:30.674821 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f92cc55-d68e-425f-acb4-cd374b407057 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81449) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:11:30.675115 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f92cc55-d68e-425f-acb4-cd374b407057 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:11:30.676459 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f92cc55-d68e-425f-acb4-cd374b407057 None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:11:30.678675 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:11:30.688478 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 326/656] 10.222.0.15 () {56 vars in 1109 bytes} [Thu Sep 15 12:11:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:11:30.689290 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f92cc55-d68e-425f-acb4-cd374b407057 None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:30.699278 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:11:30.714027 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f838b631-7023-43cc-88c3-db48fba38466 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:30.747978 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd4b390-49cd-4fc3-a3da-e549d1ab9b48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:30.748586 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd4b390-49cd-4fc3-a3da-e549d1ab9b48 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:30.748909 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd4b390-49cd-4fc3-a3da-e549d1ab9b48 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:30.749185 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd4b390-49cd-4fc3-a3da-e549d1ab9b48 None admin] PATH_INFO: `/v3/users` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:30.749907 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd4b390-49cd-4fc3-a3da-e549d1ab9b48 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:11:30.750214 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd4b390-49cd-4fc3-a3da-e549d1ab9b48 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:11:30.751623 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd4b390-49cd-4fc3-a3da-e549d1ab9b48 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:11:30.768838 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 331/657] 10.222.0.15 () {56 vars in 1137 bytes} [Thu Sep 15 12:11:30 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:11:30.769889 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd4b390-49cd-4fc3-a3da-e549d1ab9b48 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:30.792200 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f92cc55-d68e-425f-acb4-cd374b407057 None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:30.825755 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2abb9591-3913-44ff-ba1f-ad0ee6820659 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:30.826359 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abb9591-3913-44ff-ba1f-ad0ee6820659 None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:30.826641 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abb9591-3913-44ff-ba1f-ad0ee6820659 None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:30.826938 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abb9591-3913-44ff-ba1f-ad0ee6820659 None admin] PATH_INFO: `/v3/users` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:30.827644 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2abb9591-3913-44ff-ba1f-ad0ee6820659 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81449) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:11:30.827944 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2abb9591-3913-44ff-ba1f-ad0ee6820659 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:11:30.829419 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2abb9591-3913-44ff-ba1f-ad0ee6820659 None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:11:30.846746 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 327/658] 10.222.0.15 () {56 vars in 1139 bytes} [Thu Sep 15 12:11:30 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:11:30.847555 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2abb9591-3913-44ff-ba1f-ad0ee6820659 None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:30.872289 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd4b390-49cd-4fc3-a3da-e549d1ab9b48 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:30.900866 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:11:30.907096 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21cd4db8-dc61-45aa-9450-02d6c610fd6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:30.907763 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cd4db8-dc61-45aa-9450-02d6c610fd6b None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:30.908043 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cd4db8-dc61-45aa-9450-02d6c610fd6b None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:30.908314 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cd4db8-dc61-45aa-9450-02d6c610fd6b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:30.911561 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:11:30.919300 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21cd4db8-dc61-45aa-9450-02d6c610fd6b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:11:30.920742 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21cd4db8-dc61-45aa-9450-02d6c610fd6b None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:11:30.934955 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application [None req-21cd4db8-dc61-45aa-9450-02d6c610fd6b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 12:11:30.934955 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:11:30.934955 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:11:30.934955 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:11:30.934955 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:11:30.934955 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:11:30.934955 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:11:30.934955 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:11:30.934955 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:11:30.934955 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:11:30.934955 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:11:30.934955 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:11:30.934955 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 12:11:30.934955 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 12:11:30.934955 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 12:11:30.934955 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 12:11:30.935942 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:11:30.935942 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:11:30.935942 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:11:30.935942 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:11:30.935942 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:11:30.935942 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:11:30.935942 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:11:30.935942 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application with Lock( Sep 15 12:11:30.935942 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:11:30.935942 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:11:30.935942 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:11:30.935942 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:11:30.935942 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:11:30.935942 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:11:30.935942 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:11:30.935942 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:11:30.935942 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 12:11:30.936879 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 12:11:30.936879 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 12:11:30.936879 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 15 12:11:30.936879 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 332/659] 10.222.0.15 () {56 vars in 1108 bytes} [Thu Sep 15 12:11:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:11:30.936879 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21cd4db8-dc61-45aa-9450-02d6c610fd6b None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:30.950270 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2abb9591-3913-44ff-ba1f-ad0ee6820659 None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:30.984732 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5effe535-c2d3-4ad8-9bb7-3a21aad27ddb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:30.985393 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5effe535-c2d3-4ad8-9bb7-3a21aad27ddb None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:30.985674 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5effe535-c2d3-4ad8-9bb7-3a21aad27ddb None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:30.985940 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5effe535-c2d3-4ad8-9bb7-3a21aad27ddb None admin] PATH_INFO: `/v3/roles/service` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:30.995853 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5effe535-c2d3-4ad8-9bb7-3a21aad27ddb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 12:11:31.000540 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application [None req-5effe535-c2d3-4ad8-9bb7-3a21aad27ddb None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 15 12:11:31.000540 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:11:31.000540 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:11:31.000540 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:11:31.000540 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:11:31.000540 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:11:31.000540 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:11:31.000540 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:11:31.000540 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:11:31.000540 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:11:31.000540 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:11:31.000540 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:11:31.000540 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 12:11:31.000540 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 12:11:31.000540 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 12:11:31.000540 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 12:11:31.001552 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 12:11:31.001552 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 12:11:31.001552 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 12:11:31.001552 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 12:11:31.001552 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:11:31.001552 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:11:31.001552 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:11:31.001552 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:11:31.001552 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:11:31.001552 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:11:31.001552 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:11:31.001552 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application with Lock( Sep 15 12:11:31.001552 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:11:31.001552 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:11:31.001552 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:11:31.001552 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:11:31.002855 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:11:31.002855 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:11:31.002855 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:11:31.002855 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:11:31.002855 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 12:11:31.002855 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 12:11:31.002855 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 12:11:31.002855 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 12:11:31.002855 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 12:11:31.002855 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 12:11:31.002855 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 15 12:11:31.002855 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application Sep 15 12:11:31.002855 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 328/660] 10.222.0.15 () {56 vars in 1102 bytes} [Thu Sep 15 12:11:30 2022] GET /identity/v3/roles/service => generated 85 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:11:31.002855 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5effe535-c2d3-4ad8-9bb7-3a21aad27ddb None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:31.040322 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21cd4db8-dc61-45aa-9450-02d6c610fd6b None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:31.072424 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b8e85e-79f8-4edf-bea7-230be307bab5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:31.073086 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b8e85e-79f8-4edf-bea7-230be307bab5 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:31.073361 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b8e85e-79f8-4edf-bea7-230be307bab5 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:31.073628 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b8e85e-79f8-4edf-bea7-230be307bab5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:31.080640 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b8e85e-79f8-4edf-bea7-230be307bab5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:11:31.082127 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b8e85e-79f8-4edf-bea7-230be307bab5 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:11:31.093583 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application [None req-50b8e85e-79f8-4edf-bea7-230be307bab5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 12:11:31.093583 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:11:31.093583 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:11:31.093583 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:11:31.093583 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:11:31.093583 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:11:31.093583 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:11:31.093583 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:11:31.093583 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:11:31.093583 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:11:31.093583 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:11:31.093583 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:11:31.093583 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 15 12:11:31.093583 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 15 12:11:31.093583 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 15 12:11:31.094618 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 15 12:11:31.094618 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:11:31.094618 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:11:31.094618 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:11:31.094618 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:11:31.094618 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:11:31.094618 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:11:31.094618 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:11:31.094618 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application with Lock( Sep 15 12:11:31.094618 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:11:31.094618 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:11:31.094618 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:11:31.094618 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:11:31.094618 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:11:31.094618 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:11:31.094618 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:11:31.094618 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:11:31.095508 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 15 12:11:31.095508 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 15 12:11:31.095508 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 15 12:11:31.095508 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 15 12:11:31.095508 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 15 12:11:31.095508 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 15 12:11:31.095508 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 12:11:31.095508 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 15 12:11:31.095508 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 333/661] 10.222.0.15 () {56 vars in 1111 bytes} [Thu Sep 15 12:11:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:11:31.095508 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b8e85e-79f8-4edf-bea7-230be307bab5 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:31.106680 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5effe535-c2d3-4ad8-9bb7-3a21aad27ddb None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:31.131930 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:11:31.136665 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:11:31.141135 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db7c584-80ef-4b39-b960-499ad5e69959 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:31.141735 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db7c584-80ef-4b39-b960-499ad5e69959 None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:31.142010 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db7c584-80ef-4b39-b960-499ad5e69959 None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:31.142278 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db7c584-80ef-4b39-b960-499ad5e69959 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:31.194080 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b8e85e-79f8-4edf-bea7-230be307bab5 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:31.207656 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db7c584-80ef-4b39-b960-499ad5e69959 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:11:31.209169 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db7c584-80ef-4b39-b960-499ad5e69959 None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:11:31.226815 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29611405-a122-4970-ab47-b61561fd6fe6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:31.227420 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29611405-a122-4970-ab47-b61561fd6fe6 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:31.227698 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29611405-a122-4970-ab47-b61561fd6fe6 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:31.227968 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29611405-a122-4970-ab47-b61561fd6fe6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:31.295540 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29611405-a122-4970-ab47-b61561fd6fe6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:11:31.296985 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29611405-a122-4970-ab47-b61561fd6fe6 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:11:31.305565 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 329/662] 10.222.0.15 () {58 vars in 1282 bytes} [Thu Sep 15 12:11:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 15 12:11:31.306392 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db7c584-80ef-4b39-b960-499ad5e69959 None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:31.331628 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:11:31.348120 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:11:31.402354 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db7c584-80ef-4b39-b960-499ad5e69959 None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:31.405424 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 334/663] 10.222.0.15 () {58 vars in 1282 bytes} [Thu Sep 15 12:11:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 15 12:11:31.406252 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29611405-a122-4970-ab47-b61561fd6fe6 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:31.436954 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3bf8e62-457b-444e-9b04-99a7d39aaee1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:31.437563 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bf8e62-457b-444e-9b04-99a7d39aaee1 None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:31.437842 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bf8e62-457b-444e-9b04-99a7d39aaee1 None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:31.438113 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bf8e62-457b-444e-9b04-99a7d39aaee1 None admin] PATH_INFO: `/v3/domains` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:31.438746 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3bf8e62-457b-444e-9b04-99a7d39aaee1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81449) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:11:31.439050 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3bf8e62-457b-444e-9b04-99a7d39aaee1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:11:31.440404 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3bf8e62-457b-444e-9b04-99a7d39aaee1 None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:11:31.451410 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 330/664] 10.222.0.15 () {56 vars in 1109 bytes} [Thu Sep 15 12:11:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:11:31.452193 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3bf8e62-457b-444e-9b04-99a7d39aaee1 None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:31.503738 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29611405-a122-4970-ab47-b61561fd6fe6 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:31.536444 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:11:31.536595 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc3f0d0-1371-4550-9386-87d1de41e9e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:31.537198 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc3f0d0-1371-4550-9386-87d1de41e9e4 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:31.537473 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc3f0d0-1371-4550-9386-87d1de41e9e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:31.537734 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc3f0d0-1371-4550-9386-87d1de41e9e4 None admin] PATH_INFO: `/v3/roles` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:31.538378 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc3f0d0-1371-4550-9386-87d1de41e9e4 None admin] RBAC: Adding query filter params (name=service) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:11:31.538674 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc3f0d0-1371-4550-9386-87d1de41e9e4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:11:31.540031 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc3f0d0-1371-4550-9386-87d1de41e9e4 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:11:31.547886 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 335/665] 10.222.0.15 () {56 vars in 1103 bytes} [Thu Sep 15 12:11:31 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:11:31.548747 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc3f0d0-1371-4550-9386-87d1de41e9e4 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:31.552489 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:11:31.556281 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3bf8e62-457b-444e-9b04-99a7d39aaee1 None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:31.589116 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c80f6d-ac13-4278-8503-d20ec0363fbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:31.589726 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c80f6d-ac13-4278-8503-d20ec0363fbd None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:31.590003 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c80f6d-ac13-4278-8503-d20ec0363fbd None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:31.590273 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c80f6d-ac13-4278-8503-d20ec0363fbd None admin] PATH_INFO: `/v3/projects` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:31.590983 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c80f6d-ac13-4278-8503-d20ec0363fbd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81449) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:11:31.591286 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c80f6d-ac13-4278-8503-d20ec0363fbd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:11:31.592725 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c80f6d-ac13-4278-8503-d20ec0363fbd None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:11:31.605217 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 331/666] 10.222.0.15 () {56 vars in 1148 bytes} [Thu Sep 15 12:11:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:11:31.605984 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c80f6d-ac13-4278-8503-d20ec0363fbd None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:31.649537 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc3f0d0-1371-4550-9386-87d1de41e9e4 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:31.681778 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6bc17c4-63b5-44a4-bcb8-2eda7d9c81ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:31.682418 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bc17c4-63b5-44a4-bcb8-2eda7d9c81ce None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:31.682694 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bc17c4-63b5-44a4-bcb8-2eda7d9c81ce None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:31.682966 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bc17c4-63b5-44a4-bcb8-2eda7d9c81ce None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:31.693973 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6bc17c4-63b5-44a4-bcb8-2eda7d9c81ce None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:11:31.695400 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6bc17c4-63b5-44a4-bcb8-2eda7d9c81ce None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:11:31.703967 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c80f6d-ac13-4278-8503-d20ec0363fbd None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:31.710142 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application [None req-a6bc17c4-63b5-44a4-bcb8-2eda7d9c81ce None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 12:11:31.710142 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:11:31.710142 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:11:31.710142 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:11:31.710142 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:11:31.710142 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:11:31.710142 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:11:31.710142 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:11:31.710142 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:11:31.710142 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:11:31.710142 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:11:31.710142 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:11:31.710142 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 12:11:31.710142 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 12:11:31.710142 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 12:11:31.710142 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 12:11:31.711087 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:11:31.711087 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:11:31.711087 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:11:31.711087 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:11:31.711087 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:11:31.711087 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:11:31.711087 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:11:31.711087 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application with Lock( Sep 15 12:11:31.711087 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:11:31.711087 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:11:31.711087 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:11:31.711087 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:11:31.711087 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:11:31.711087 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:11:31.711087 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:11:31.711087 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:11:31.711087 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 12:11:31.712019 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 12:11:31.712019 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 12:11:31.712019 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 15 12:11:31.712019 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 336/667] 10.222.0.15 () {56 vars in 1108 bytes} [Thu Sep 15 12:11:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:11:31.712019 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6bc17c4-63b5-44a4-bcb8-2eda7d9c81ce None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:31.743387 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c612d3bc-c4a9-41b6-84cb-2eac4d21ccde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:31.744017 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c612d3bc-c4a9-41b6-84cb-2eac4d21ccde None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:31.744298 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c612d3bc-c4a9-41b6-84cb-2eac4d21ccde None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:31.744571 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c612d3bc-c4a9-41b6-84cb-2eac4d21ccde None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:31.754419 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c612d3bc-c4a9-41b6-84cb-2eac4d21ccde None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:11:31.756443 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c612d3bc-c4a9-41b6-84cb-2eac4d21ccde None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:11:31.763088 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:11:31.767005 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:11:31.771162 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application [None req-c612d3bc-c4a9-41b6-84cb-2eac4d21ccde None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 12:11:31.771162 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:11:31.771162 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:11:31.771162 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:11:31.771162 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:11:31.771162 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:11:31.771162 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:11:31.771162 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:11:31.771162 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:11:31.771162 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:11:31.771162 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:11:31.771162 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:11:31.771162 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 12:11:31.771162 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 12:11:31.771162 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 12:11:31.771162 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 12:11:31.772120 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:11:31.772120 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:11:31.772120 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:11:31.772120 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:11:31.772120 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:11:31.772120 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:11:31.772120 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:11:31.772120 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application with Lock( Sep 15 12:11:31.772120 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:11:31.772120 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:11:31.772120 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:11:31.772120 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:11:31.772120 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:11:31.772120 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:11:31.772120 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:11:31.772120 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:11:31.772120 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 12:11:31.773532 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 12:11:31.773532 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 12:11:31.773532 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application Sep 15 12:11:31.773532 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 332/668] 10.222.0.15 () {56 vars in 1108 bytes} [Thu Sep 15 12:11:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:11:31.773532 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c612d3bc-c4a9-41b6-84cb-2eac4d21ccde None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:31.814434 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6bc17c4-63b5-44a4-bcb8-2eda7d9c81ce None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:31.846882 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c752a4-88ed-461e-ad56-eed1195a3512 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:31.847514 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c752a4-88ed-461e-ad56-eed1195a3512 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:31.847792 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c752a4-88ed-461e-ad56-eed1195a3512 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:31.848089 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c752a4-88ed-461e-ad56-eed1195a3512 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:31.854900 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c752a4-88ed-461e-ad56-eed1195a3512 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:11:31.856327 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c752a4-88ed-461e-ad56-eed1195a3512 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:11:31.868096 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application [None req-57c752a4-88ed-461e-ad56-eed1195a3512 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 12:11:31.868096 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:11:31.868096 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:11:31.868096 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:11:31.868096 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:11:31.868096 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:11:31.868096 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:11:31.868096 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:11:31.868096 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:11:31.868096 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:11:31.868096 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:11:31.868096 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:11:31.868096 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 15 12:11:31.868096 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 15 12:11:31.868096 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 15 12:11:31.869183 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 15 12:11:31.869183 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:11:31.869183 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:11:31.869183 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:11:31.869183 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:11:31.869183 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:11:31.869183 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:11:31.869183 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:11:31.869183 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application with Lock( Sep 15 12:11:31.869183 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:11:31.869183 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:11:31.869183 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:11:31.869183 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:11:31.869183 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:11:31.869183 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:11:31.869183 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:11:31.869183 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:11:31.870278 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 15 12:11:31.870278 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 15 12:11:31.870278 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 15 12:11:31.870278 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 15 12:11:31.870278 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 15 12:11:31.870278 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 15 12:11:31.870278 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 12:11:31.870278 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 15 12:11:31.870278 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 337/669] 10.222.0.15 () {56 vars in 1111 bytes} [Thu Sep 15 12:11:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:11:31.870849 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c752a4-88ed-461e-ad56-eed1195a3512 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:31.874731 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c612d3bc-c4a9-41b6-84cb-2eac4d21ccde None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:31.907662 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b380ab07-b5e1-40db-83f0-6c2c3ac9d243 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:31.908320 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b380ab07-b5e1-40db-83f0-6c2c3ac9d243 None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:31.908599 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b380ab07-b5e1-40db-83f0-6c2c3ac9d243 None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:31.908899 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b380ab07-b5e1-40db-83f0-6c2c3ac9d243 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:31.919160 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b380ab07-b5e1-40db-83f0-6c2c3ac9d243 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:11:31.920675 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b380ab07-b5e1-40db-83f0-6c2c3ac9d243 None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:11:31.935341 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application [None req-b380ab07-b5e1-40db-83f0-6c2c3ac9d243 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 12:11:31.935341 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:11:31.935341 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:11:31.935341 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:11:31.935341 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:11:31.935341 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:11:31.935341 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:11:31.935341 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:11:31.935341 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:11:31.935341 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:11:31.935341 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:11:31.935341 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:11:31.935341 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 12:11:31.935341 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 12:11:31.935341 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 12:11:31.935341 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 12:11:31.936333 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:11:31.936333 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:11:31.936333 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:11:31.936333 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:11:31.936333 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:11:31.936333 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:11:31.936333 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:11:31.936333 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application with Lock( Sep 15 12:11:31.936333 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:11:31.936333 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:11:31.936333 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:11:31.936333 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:11:31.936333 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:11:31.936333 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:11:31.936333 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:11:31.936333 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:11:31.936333 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 12:11:31.937630 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 12:11:31.937630 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 12:11:31.937630 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application Sep 15 12:11:31.937630 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 333/670] 10.222.0.15 () {56 vars in 1108 bytes} [Thu Sep 15 12:11:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:11:31.937630 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b380ab07-b5e1-40db-83f0-6c2c3ac9d243 None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:31.976360 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c752a4-88ed-461e-ad56-eed1195a3512 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:31.986416 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:11:31.993161 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:11:32.012086 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b46b6d8-3ff7-4be1-ab51-b2988f0572b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:32.012697 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b46b6d8-3ff7-4be1-ab51-b2988f0572b8 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:32.013012 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b46b6d8-3ff7-4be1-ab51-b2988f0572b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:32.013287 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b46b6d8-3ff7-4be1-ab51-b2988f0572b8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:32.013997 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b46b6d8-3ff7-4be1-ab51-b2988f0572b8 None admin] RBAC: Adding query filter params (role.id=e1cecc29e028497f9cdaa04268c6a6ec, scope.project.id=540a32ef88764e8c8d9201ae22f6ddaf, user.id=eface89e261840a2abb4206be26455c7) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:11:32.014301 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b46b6d8-3ff7-4be1-ab51-b2988f0572b8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:11:32.015718 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b46b6d8-3ff7-4be1-ab51-b2988f0572b8 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:11:32.021324 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 338/671] 10.222.0.15 () {56 vars in 1374 bytes} [Thu Sep 15 12:11:31 2022] GET /identity/v3/role_assignments?user.id=eface89e261840a2abb4206be26455c7&scope.project.id=540a32ef88764e8c8d9201ae22f6ddaf&role.id=e1cecc29e028497f9cdaa04268c6a6ec => generated 608 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:11:32.022128 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b46b6d8-3ff7-4be1-ab51-b2988f0572b8 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:32.038252 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b380ab07-b5e1-40db-83f0-6c2c3ac9d243 None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:32.085585 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37b2bd6-d07a-4f7b-be46-d51eb262b5ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:32.086420 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37b2bd6-d07a-4f7b-be46-d51eb262b5ff None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:32.086702 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37b2bd6-d07a-4f7b-be46-d51eb262b5ff None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:32.086973 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37b2bd6-d07a-4f7b-be46-d51eb262b5ff None admin] PATH_INFO: `/v3/domains` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:32.087624 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37b2bd6-d07a-4f7b-be46-d51eb262b5ff None admin] RBAC: Adding query filter params (name=Default) {{(pid=81449) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:11:32.087931 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37b2bd6-d07a-4f7b-be46-d51eb262b5ff None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:11:32.090168 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f37b2bd6-d07a-4f7b-be46-d51eb262b5ff None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:11:32.107476 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 334/672] 10.222.0.15 () {56 vars in 1109 bytes} [Thu Sep 15 12:11:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:11:32.108155 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37b2bd6-d07a-4f7b-be46-d51eb262b5ff None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:32.148323 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b46b6d8-3ff7-4be1-ab51-b2988f0572b8 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:32.186339 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd559092-d0a0-4553-97d7-439c05d13d4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:32.187214 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd559092-d0a0-4553-97d7-439c05d13d4d None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:32.187671 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd559092-d0a0-4553-97d7-439c05d13d4d None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:32.187948 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd559092-d0a0-4553-97d7-439c05d13d4d None admin] PATH_INFO: `/v3/domains` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:32.189025 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd559092-d0a0-4553-97d7-439c05d13d4d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:11:32.189025 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd559092-d0a0-4553-97d7-439c05d13d4d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:11:32.190288 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd559092-d0a0-4553-97d7-439c05d13d4d None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:11:32.202471 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 339/673] 10.222.0.15 () {56 vars in 1109 bytes} [Thu Sep 15 12:11:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:11:32.204187 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd559092-d0a0-4553-97d7-439c05d13d4d None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:32.234821 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37b2bd6-d07a-4f7b-be46-d51eb262b5ff None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:32.250810 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:11:32.253090 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:11:32.280859 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c2c22e-7186-471e-9b69-6b44e154e71c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:32.281499 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c2c22e-7186-471e-9b69-6b44e154e71c None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:32.281780 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c2c22e-7186-471e-9b69-6b44e154e71c None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:32.282051 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c2c22e-7186-471e-9b69-6b44e154e71c None admin] PATH_INFO: `/v3/projects` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:32.282746 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c2c22e-7186-471e-9b69-6b44e154e71c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81449) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:11:32.283050 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c2c22e-7186-471e-9b69-6b44e154e71c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:11:32.284475 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c2c22e-7186-471e-9b69-6b44e154e71c None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:11:32.297128 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 335/674] 10.222.0.15 () {56 vars in 1148 bytes} [Thu Sep 15 12:11:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:11:32.297956 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c2c22e-7186-471e-9b69-6b44e154e71c None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:32.332368 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd559092-d0a0-4553-97d7-439c05d13d4d None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:32.371199 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4af2ab-62fe-4c94-acae-da064b9faabb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:32.371812 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4af2ab-62fe-4c94-acae-da064b9faabb None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:32.372087 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4af2ab-62fe-4c94-acae-da064b9faabb None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:32.372352 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4af2ab-62fe-4c94-acae-da064b9faabb None admin] PATH_INFO: `/v3/domains` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:32.373021 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4af2ab-62fe-4c94-acae-da064b9faabb None admin] RBAC: Adding query filter params (name=Default) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:11:32.373316 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4af2ab-62fe-4c94-acae-da064b9faabb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:11:32.374665 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4af2ab-62fe-4c94-acae-da064b9faabb None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:11:32.391478 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 340/675] 10.222.0.15 () {56 vars in 1109 bytes} [Thu Sep 15 12:11:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:11:32.391478 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4af2ab-62fe-4c94-acae-da064b9faabb None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:32.404426 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c2c22e-7186-471e-9b69-6b44e154e71c None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:32.442925 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1690f94-ad09-4baf-9ccb-c177aecbb4c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:32.443480 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1690f94-ad09-4baf-9ccb-c177aecbb4c6 None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:32.443761 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1690f94-ad09-4baf-9ccb-c177aecbb4c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:32.444031 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1690f94-ad09-4baf-9ccb-c177aecbb4c6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:32.451863 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1690f94-ad09-4baf-9ccb-c177aecbb4c6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:11:32.453248 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1690f94-ad09-4baf-9ccb-c177aecbb4c6 None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:11:32.464864 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application [None req-c1690f94-ad09-4baf-9ccb-c177aecbb4c6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 12:11:32.464864 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:11:32.464864 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:11:32.464864 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:11:32.464864 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:11:32.464864 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:11:32.464864 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:11:32.464864 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:11:32.464864 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:11:32.464864 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:11:32.464864 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:11:32.464864 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:11:32.464864 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 15 12:11:32.464864 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 15 12:11:32.464864 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 15 12:11:32.465868 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 15 12:11:32.465868 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:11:32.465868 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:11:32.465868 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:11:32.465868 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:11:32.465868 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:11:32.465868 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:11:32.465868 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:11:32.465868 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application with Lock( Sep 15 12:11:32.465868 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:11:32.465868 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:11:32.465868 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:11:32.465868 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:11:32.465868 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:11:32.465868 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:11:32.465868 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:11:32.465868 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:11:32.466767 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 15 12:11:32.466767 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 15 12:11:32.466767 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 15 12:11:32.466767 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 15 12:11:32.466767 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 15 12:11:32.466767 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 15 12:11:32.466767 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 12:11:32.466767 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application Sep 15 12:11:32.466767 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 336/676] 10.222.0.15 () {56 vars in 1111 bytes} [Thu Sep 15 12:11:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:11:32.466767 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1690f94-ad09-4baf-9ccb-c177aecbb4c6 None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:32.498468 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:11:32.500110 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:11:32.502869 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4af2ab-62fe-4c94-acae-da064b9faabb None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:32.540549 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5b53e4-66dc-4257-a45a-94ef2635ff28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:32.540549 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5b53e4-66dc-4257-a45a-94ef2635ff28 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:32.541006 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5b53e4-66dc-4257-a45a-94ef2635ff28 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:32.541167 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5b53e4-66dc-4257-a45a-94ef2635ff28 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:32.547941 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c5b53e4-66dc-4257-a45a-94ef2635ff28 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:11:32.549457 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c5b53e4-66dc-4257-a45a-94ef2635ff28 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:11:32.561335 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application [None req-9c5b53e4-66dc-4257-a45a-94ef2635ff28 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 12:11:32.561335 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:11:32.561335 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:11:32.561335 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:11:32.561335 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:11:32.561335 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:11:32.561335 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:11:32.561335 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:11:32.561335 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:11:32.561335 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:11:32.561335 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:11:32.561335 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:11:32.561335 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 15 12:11:32.561335 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 15 12:11:32.561335 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 15 12:11:32.562434 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 15 12:11:32.562434 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:11:32.562434 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:11:32.562434 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:11:32.562434 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:11:32.562434 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:11:32.562434 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:11:32.562434 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:11:32.562434 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application with Lock( Sep 15 12:11:32.562434 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:11:32.562434 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:11:32.562434 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:11:32.562434 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:11:32.562434 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:11:32.562434 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:11:32.562434 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:11:32.562434 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:11:32.563687 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 15 12:11:32.563687 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 15 12:11:32.563687 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 15 12:11:32.563687 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 15 12:11:32.563687 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 15 12:11:32.563687 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 15 12:11:32.563687 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 12:11:32.563687 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 15 12:11:32.563687 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 341/677] 10.222.0.15 () {56 vars in 1111 bytes} [Thu Sep 15 12:11:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:11:32.563687 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5b53e4-66dc-4257-a45a-94ef2635ff28 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:32.577225 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1690f94-ad09-4baf-9ccb-c177aecbb4c6 None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:32.614584 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d23b46-37fa-4662-9355-65b049d48370 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:32.615195 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d23b46-37fa-4662-9355-65b049d48370 None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:32.615470 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d23b46-37fa-4662-9355-65b049d48370 None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:32.615738 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d23b46-37fa-4662-9355-65b049d48370 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:32.616464 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d23b46-37fa-4662-9355-65b049d48370 None admin] RBAC: Adding query filter params (role.id=e1cecc29e028497f9cdaa04268c6a6ec, scope.project.id=540a32ef88764e8c8d9201ae22f6ddaf, user.id=c8191d3ccb874529b32f8d0026db629d) {{(pid=81449) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:11:32.616769 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d23b46-37fa-4662-9355-65b049d48370 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:11:32.618203 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d23b46-37fa-4662-9355-65b049d48370 None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:11:32.624572 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 337/678] 10.222.0.15 () {56 vars in 1374 bytes} [Thu Sep 15 12:11:32 2022] GET /identity/v3/role_assignments?user.id=c8191d3ccb874529b32f8d0026db629d&scope.project.id=540a32ef88764e8c8d9201ae22f6ddaf&role.id=e1cecc29e028497f9cdaa04268c6a6ec => generated 608 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:11:32.625144 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d23b46-37fa-4662-9355-65b049d48370 None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:32.704899 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5b53e4-66dc-4257-a45a-94ef2635ff28 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:32.741448 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:11:32.750239 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d23b46-37fa-4662-9355-65b049d48370 None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:32.758114 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032193ec-5325-44e5-92d3-08a950b8991e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:32.759225 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032193ec-5325-44e5-92d3-08a950b8991e None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:32.760514 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032193ec-5325-44e5-92d3-08a950b8991e None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:32.761101 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032193ec-5325-44e5-92d3-08a950b8991e None admin] PATH_INFO: `/v3/users/swift` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:32.773985 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032193ec-5325-44e5-92d3-08a950b8991e None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:11:32.776944 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032193ec-5325-44e5-92d3-08a950b8991e None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:11:32.791597 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52213c21-c862-449c-adb5-46e86c2602a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:32.792686 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52213c21-c862-449c-adb5-46e86c2602a3 None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:32.793081 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52213c21-c862-449c-adb5-46e86c2602a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:32.793358 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52213c21-c862-449c-adb5-46e86c2602a3 None admin] PATH_INFO: `/v3/projects` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:32.794064 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52213c21-c862-449c-adb5-46e86c2602a3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81449) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:11:32.794368 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52213c21-c862-449c-adb5-46e86c2602a3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:11:32.795837 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application [None req-032193ec-5325-44e5-92d3-08a950b8991e None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 15 12:11:32.795837 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:11:32.795837 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:11:32.795837 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:11:32.795837 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:11:32.795837 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:11:32.795837 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:11:32.795837 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:11:32.795837 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:11:32.795837 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:11:32.795837 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:11:32.795837 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:11:32.795837 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 15 12:11:32.795837 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 15 12:11:32.795837 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 15 12:11:32.795837 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 15 12:11:32.796789 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:11:32.796789 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:11:32.796789 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 15 12:11:32.796789 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 12:11:32.796789 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 15 12:11:32.796789 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 12:11:32.796789 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:11:32.796789 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:11:32.796789 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:11:32.796789 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:11:32.796789 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:11:32.796789 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application with Lock( Sep 15 12:11:32.796789 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:11:32.796789 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:11:32.796789 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:11:32.796789 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:11:32.796789 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:11:32.796789 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:11:32.797832 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:11:32.797832 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:11:32.797832 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 15 12:11:32.797832 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 15 12:11:32.797832 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 15 12:11:32.797832 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 15 12:11:32.797832 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 15 12:11:32.797832 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 15 12:11:32.797832 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 15 12:11:32.797832 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 15 12:11:32.798379 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52213c21-c862-449c-adb5-46e86c2602a3 None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:11:32.800109 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 342/679] 10.222.0.15 () {56 vars in 1096 bytes} [Thu Sep 15 12:11:32 2022] GET /identity/v3/users/swift => generated 83 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:11:32.802485 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032193ec-5325-44e5-92d3-08a950b8991e None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:32.817795 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 338/680] 10.222.0.15 () {56 vars in 1148 bytes} [Thu Sep 15 12:11:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:11:32.817795 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52213c21-c862-449c-adb5-46e86c2602a3 None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:32.837147 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:11:32.928038 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032193ec-5325-44e5-92d3-08a950b8991e None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:32.947452 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52213c21-c862-449c-adb5-46e86c2602a3 None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:32.969017 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb88ab08-a4d0-4a77-b7f8-8ce6d036f002 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:32.969567 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb88ab08-a4d0-4a77-b7f8-8ce6d036f002 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:32.969849 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb88ab08-a4d0-4a77-b7f8-8ce6d036f002 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:32.970121 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb88ab08-a4d0-4a77-b7f8-8ce6d036f002 None admin] PATH_INFO: `/v3/projects` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:32.970932 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb88ab08-a4d0-4a77-b7f8-8ce6d036f002 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:11:32.971311 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb88ab08-a4d0-4a77-b7f8-8ce6d036f002 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:11:32.972821 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb88ab08-a4d0-4a77-b7f8-8ce6d036f002 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:11:32.987370 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 343/681] 10.222.0.15 () {56 vars in 1148 bytes} [Thu Sep 15 12:11:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:11:32.988243 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb88ab08-a4d0-4a77-b7f8-8ce6d036f002 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:32.997237 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c835a7-7893-4902-8226-28f764c6f79f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:32.997525 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c835a7-7893-4902-8226-28f764c6f79f None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:32.997809 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c835a7-7893-4902-8226-28f764c6f79f None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:32.998082 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27c835a7-7893-4902-8226-28f764c6f79f None admin] PATH_INFO: `/v3/users` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:32.998812 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c835a7-7893-4902-8226-28f764c6f79f None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81449) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:11:32.999117 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c835a7-7893-4902-8226-28f764c6f79f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:11:33.000563 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27c835a7-7893-4902-8226-28f764c6f79f None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:11:33.018997 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 339/682] 10.222.0.15 () {56 vars in 1135 bytes} [Thu Sep 15 12:11:32 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:11:33.019930 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c835a7-7893-4902-8226-28f764c6f79f None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:33.035172 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:11:33.096588 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:11:33.119662 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb88ab08-a4d0-4a77-b7f8-8ce6d036f002 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:33.125746 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27c835a7-7893-4902-8226-28f764c6f79f None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:33.159806 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2689e143-d174-4855-b81f-00a3f39d8d7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:33.160898 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2689e143-d174-4855-b81f-00a3f39d8d7c None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:33.161011 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79b0992-2d96-4b98-9bc4-b57cfca78596 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:33.161519 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2689e143-d174-4855-b81f-00a3f39d8d7c None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:33.161699 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79b0992-2d96-4b98-9bc4-b57cfca78596 None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:33.161971 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79b0992-2d96-4b98-9bc4-b57cfca78596 None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:33.162247 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79b0992-2d96-4b98-9bc4-b57cfca78596 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:33.162584 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2689e143-d174-4855-b81f-00a3f39d8d7c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:33.162960 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79b0992-2d96-4b98-9bc4-b57cfca78596 None admin] RBAC: Adding query filter params (role.id=e1cecc29e028497f9cdaa04268c6a6ec, scope.project.id=540a32ef88764e8c8d9201ae22f6ddaf, user.id=bfbb4d6809064f9cb9e06626ed7d6814) {{(pid=81449) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:11:33.163262 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79b0992-2d96-4b98-9bc4-b57cfca78596 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:11:33.163952 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2689e143-d174-4855-b81f-00a3f39d8d7c None admin] RBAC: Adding query filter params (role.id=e1cecc29e028497f9cdaa04268c6a6ec, scope.project.id=540a32ef88764e8c8d9201ae22f6ddaf, user.id=3aed8f78903f44cda9e5254aa46cf2b7) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:11:33.164472 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2689e143-d174-4855-b81f-00a3f39d8d7c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:11:33.164778 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79b0992-2d96-4b98-9bc4-b57cfca78596 None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:11:33.166308 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2689e143-d174-4855-b81f-00a3f39d8d7c None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:11:33.170870 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 340/683] 10.222.0.15 () {56 vars in 1374 bytes} [Thu Sep 15 12:11:33 2022] GET /identity/v3/role_assignments?user.id=bfbb4d6809064f9cb9e06626ed7d6814&scope.project.id=540a32ef88764e8c8d9201ae22f6ddaf&role.id=e1cecc29e028497f9cdaa04268c6a6ec => generated 608 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:11:33.171442 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79b0992-2d96-4b98-9bc4-b57cfca78596 None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:33.180048 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 344/684] 10.222.0.15 () {56 vars in 1374 bytes} [Thu Sep 15 12:11:32 2022] GET /identity/v3/role_assignments?user.id=3aed8f78903f44cda9e5254aa46cf2b7&scope.project.id=540a32ef88764e8c8d9201ae22f6ddaf&role.id=e1cecc29e028497f9cdaa04268c6a6ec => generated 608 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:11:33.323544 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:11:33.366581 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79b0992-2d96-4b98-9bc4-b57cfca78596 None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:33.411564 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b44a3d4-99d0-4203-bb25-612dbb0ce475 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:33.412199 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b44a3d4-99d0-4203-bb25-612dbb0ce475 None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:33.412530 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b44a3d4-99d0-4203-bb25-612dbb0ce475 None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:33.412824 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b44a3d4-99d0-4203-bb25-612dbb0ce475 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:33.493309 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b44a3d4-99d0-4203-bb25-612dbb0ce475 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:11:33.494770 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b44a3d4-99d0-4203-bb25-612dbb0ce475 None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:11:33.582728 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:11:33.610101 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 341/685] 10.222.0.15 () {58 vars in 1282 bytes} [Thu Sep 15 12:11:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 439 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 15 12:11:33.615890 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2689e143-d174-4855-b81f-00a3f39d8d7c None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:33.737217 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2689e143-d174-4855-b81f-00a3f39d8d7c None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:33.772291 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ff9e25-a414-4ae2-9ee8-3f1dd72e47af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:33.772964 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ff9e25-a414-4ae2-9ee8-3f1dd72e47af None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:33.773246 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ff9e25-a414-4ae2-9ee8-3f1dd72e47af None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:33.773516 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ff9e25-a414-4ae2-9ee8-3f1dd72e47af None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:33.784550 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ff9e25-a414-4ae2-9ee8-3f1dd72e47af None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:11:33.786554 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ff9e25-a414-4ae2-9ee8-3f1dd72e47af None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:11:33.789130 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:11:33.802244 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application [None req-20ff9e25-a414-4ae2-9ee8-3f1dd72e47af None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 12:11:33.802244 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:11:33.802244 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:11:33.802244 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:11:33.802244 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:11:33.802244 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:11:33.802244 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:11:33.802244 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:11:33.802244 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:11:33.802244 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:11:33.802244 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:11:33.802244 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:11:33.802244 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 12:11:33.802244 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 12:11:33.802244 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 12:11:33.802244 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 12:11:33.803224 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:11:33.803224 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:11:33.803224 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:11:33.803224 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:11:33.803224 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:11:33.803224 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:11:33.803224 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:11:33.803224 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application with Lock( Sep 15 12:11:33.803224 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:11:33.803224 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:11:33.803224 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:11:33.803224 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:11:33.803224 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:11:33.803224 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:11:33.803224 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:11:33.803224 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:11:33.803224 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 12:11:33.804120 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 12:11:33.804120 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 12:11:33.804120 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 15 12:11:33.804290 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 345/686] 10.222.0.15 () {56 vars in 1108 bytes} [Thu Sep 15 12:11:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:11:33.809314 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b44a3d4-99d0-4203-bb25-612dbb0ce475 None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:33.994568 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b44a3d4-99d0-4203-bb25-612dbb0ce475 None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:34.042306 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4216f2ba-3acc-4e79-8efe-91d117cfabe7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:34.043342 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4216f2ba-3acc-4e79-8efe-91d117cfabe7 None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:34.043843 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4216f2ba-3acc-4e79-8efe-91d117cfabe7 None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:34.044325 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4216f2ba-3acc-4e79-8efe-91d117cfabe7 None admin] PATH_INFO: `/v3/domains` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:34.045210 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4216f2ba-3acc-4e79-8efe-91d117cfabe7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81449) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:11:34.045713 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4216f2ba-3acc-4e79-8efe-91d117cfabe7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:11:34.047274 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4216f2ba-3acc-4e79-8efe-91d117cfabe7 None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:11:34.065887 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 342/687] 10.222.0.15 () {56 vars in 1109 bytes} [Thu Sep 15 12:11:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:11:34.071408 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ff9e25-a414-4ae2-9ee8-3f1dd72e47af None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:34.208327 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ff9e25-a414-4ae2-9ee8-3f1dd72e47af None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:34.248510 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e3d8fd-13ab-458b-b587-ef78a926ea30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:34.249523 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e3d8fd-13ab-458b-b587-ef78a926ea30 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:34.250004 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e3d8fd-13ab-458b-b587-ef78a926ea30 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:34.250493 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e3d8fd-13ab-458b-b587-ef78a926ea30 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:34.260170 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e3d8fd-13ab-458b-b587-ef78a926ea30 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:11:34.261882 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e3d8fd-13ab-458b-b587-ef78a926ea30 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:11:34.276740 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application [None req-71e3d8fd-13ab-458b-b587-ef78a926ea30 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 12:11:34.276740 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:11:34.276740 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:11:34.276740 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:11:34.276740 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:11:34.276740 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:11:34.276740 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:11:34.276740 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:11:34.276740 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:11:34.276740 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:11:34.276740 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:11:34.276740 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:11:34.276740 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 15 12:11:34.276740 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 15 12:11:34.276740 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 15 12:11:34.277749 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 15 12:11:34.277749 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:11:34.277749 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:11:34.277749 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:11:34.277749 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:11:34.277749 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:11:34.277749 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:11:34.277749 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:11:34.277749 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application with Lock( Sep 15 12:11:34.277749 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:11:34.277749 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:11:34.277749 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:11:34.277749 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:11:34.277749 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:11:34.277749 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:11:34.277749 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:11:34.277749 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:11:34.278646 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 15 12:11:34.278646 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 15 12:11:34.278646 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 15 12:11:34.278646 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 15 12:11:34.278646 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 15 12:11:34.278646 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 15 12:11:34.278646 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 12:11:34.278646 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 15 12:11:34.280793 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 346/688] 10.222.0.15 () {56 vars in 1111 bytes} [Thu Sep 15 12:11:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:11:34.287100 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4216f2ba-3acc-4e79-8efe-91d117cfabe7 None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:34.338453 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:11:34.440519 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4216f2ba-3acc-4e79-8efe-91d117cfabe7 None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:34.486149 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f75d341-5c6f-4aa9-b56f-4abcf6405af4 None None] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:34.486779 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f75d341-5c6f-4aa9-b56f-4abcf6405af4 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:34.487256 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f75d341-5c6f-4aa9-b56f-4abcf6405af4 None None] PATH_INFO: `/` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:34.489894 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 347/689] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 12:11:34 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:11:34.492815 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2dd66d-54fa-45de-bcc6-27371eeed060 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:34.493357 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2dd66d-54fa-45de-bcc6-27371eeed060 None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:34.494055 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2dd66d-54fa-45de-bcc6-27371eeed060 None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:34.494055 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2dd66d-54fa-45de-bcc6-27371eeed060 None admin] PATH_INFO: `/v3/projects` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:34.494565 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2dd66d-54fa-45de-bcc6-27371eeed060 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81449) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:11:34.494878 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2dd66d-54fa-45de-bcc6-27371eeed060 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:11:34.496335 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2dd66d-54fa-45de-bcc6-27371eeed060 None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:11:34.496552 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296a2b40-b28c-4b28-bb7a-1311ab4af35d None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:34.496877 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296a2b40-b28c-4b28-bb7a-1311ab4af35d None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:34.497156 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296a2b40-b28c-4b28-bb7a-1311ab4af35d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:34.515053 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 343/690] 10.222.0.15 () {56 vars in 1148 bytes} [Thu Sep 15 12:11:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:11:34.520953 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2dd66d-54fa-45de-bcc6-27371eeed060 None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:34.525018 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-296a2b40-b28c-4b28-bb7a-1311ab4af35d None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:11:34.526558 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-296a2b40-b28c-4b28-bb7a-1311ab4af35d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:11:34.586311 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:11:34.604026 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 348/691] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 12:11:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 15 12:11:34.658418 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:11:34.664746 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2dd66d-54fa-45de-bcc6-27371eeed060 None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:34.701087 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3483b9-773c-4d59-a385-bf2070158af8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:34.701695 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3483b9-773c-4d59-a385-bf2070158af8 None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:34.701975 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3483b9-773c-4d59-a385-bf2070158af8 None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:34.702248 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3483b9-773c-4d59-a385-bf2070158af8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:34.702960 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3483b9-773c-4d59-a385-bf2070158af8 None admin] RBAC: Adding query filter params (role.id=e1cecc29e028497f9cdaa04268c6a6ec, scope.project.id=540a32ef88764e8c8d9201ae22f6ddaf, user.id=efa0d3dc5b7540d59fdd59bbf64a04b6) {{(pid=81449) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:11:34.703266 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3483b9-773c-4d59-a385-bf2070158af8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:11:34.704717 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3483b9-773c-4d59-a385-bf2070158af8 None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:11:34.711556 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 344/692] 10.222.0.15 () {56 vars in 1374 bytes} [Thu Sep 15 12:11:34 2022] GET /identity/v3/role_assignments?user.id=efa0d3dc5b7540d59fdd59bbf64a04b6&scope.project.id=540a32ef88764e8c8d9201ae22f6ddaf&role.id=e1cecc29e028497f9cdaa04268c6a6ec => generated 608 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 12:11:34.933089 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296a2b40-b28c-4b28-bb7a-1311ab4af35d None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:35.039775 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296a2b40-b28c-4b28-bb7a-1311ab4af35d None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:35.078075 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64bee304-fe31-4a1b-bcf5-ba9737666fa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:35.078725 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bee304-fe31-4a1b-bcf5-ba9737666fa3 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:35.079008 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bee304-fe31-4a1b-bcf5-ba9737666fa3 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:35.079279 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64bee304-fe31-4a1b-bcf5-ba9737666fa3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:35.091986 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64bee304-fe31-4a1b-bcf5-ba9737666fa3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 12:11:35.097030 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application [None req-64bee304-fe31-4a1b-bcf5-ba9737666fa3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 15 12:11:35.097030 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:11:35.097030 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:11:35.097030 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:11:35.097030 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:11:35.097030 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:11:35.097030 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:11:35.097030 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:11:35.097030 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:11:35.097030 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:11:35.097030 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:11:35.097030 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:11:35.097030 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 12:11:35.097030 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 12:11:35.097030 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 12:11:35.097030 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 12:11:35.097969 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 12:11:35.097969 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 12:11:35.097969 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 12:11:35.097969 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 12:11:35.097969 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:11:35.097969 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:11:35.097969 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:11:35.097969 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:11:35.097969 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:11:35.097969 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:11:35.097969 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:11:35.097969 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application with Lock( Sep 15 12:11:35.097969 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:11:35.097969 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:11:35.097969 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:11:35.097969 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:11:35.099388 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:11:35.099388 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:11:35.099388 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:11:35.099388 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:11:35.099388 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 12:11:35.099388 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 12:11:35.099388 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 12:11:35.099388 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 12:11:35.099388 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 12:11:35.099388 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 12:11:35.099388 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 15 12:11:35.099388 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 15 12:11:35.099388 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 349/693] 10.222.0.15 () {56 vars in 1096 bytes} [Thu Sep 15 12:11:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:11:35.109872 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3483b9-773c-4d59-a385-bf2070158af8 None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:35.226611 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3483b9-773c-4d59-a385-bf2070158af8 None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:35.263988 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6bfad6-d8ab-4101-adff-0837fbea7548 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:35.265031 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6bfad6-d8ab-4101-adff-0837fbea7548 None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:35.265538 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6bfad6-d8ab-4101-adff-0837fbea7548 None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:35.266007 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6bfad6-d8ab-4101-adff-0837fbea7548 None admin] PATH_INFO: `/v3/roles` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:35.267057 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6bfad6-d8ab-4101-adff-0837fbea7548 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81449) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:11:35.267569 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6bfad6-d8ab-4101-adff-0837fbea7548 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:11:35.269213 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6bfad6-d8ab-4101-adff-0837fbea7548 None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:11:35.278419 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 345/694] 10.222.0.15 () {56 vars in 1099 bytes} [Thu Sep 15 12:11:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:11:35.284795 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64bee304-fe31-4a1b-bcf5-ba9737666fa3 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:35.307567 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c9a44e-397e-4bec-a7cd-4fa5a966d021 None None] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:35.308173 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c9a44e-397e-4bec-a7cd-4fa5a966d021 None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:35.308660 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c9a44e-397e-4bec-a7cd-4fa5a966d021 None None] PATH_INFO: `/` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:35.310599 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 346/695] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 12:11:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 15 12:11:35.315425 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7864499-0ca9-40e7-b7c4-26231b8d2c91 None None] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:35.315967 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7864499-0ca9-40e7-b7c4-26231b8d2c91 None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:35.316454 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7864499-0ca9-40e7-b7c4-26231b8d2c91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:35.343796 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.auth.core [None req-d7864499-0ca9-40e7-b7c4-26231b8d2c91 None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81449) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:11:35.345389 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-d7864499-0ca9-40e7-b7c4-26231b8d2c91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:11:35.354925 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:11:35.367203 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:11:35.427149 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64bee304-fe31-4a1b-bcf5-ba9737666fa3 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:35.452557 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 347/696] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 12:11:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 15 12:11:35.471857 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9124ba8-91fd-4fd3-9bd7-baf63cc5b12e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:35.472490 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9124ba8-91fd-4fd3-9bd7-baf63cc5b12e None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:35.472782 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9124ba8-91fd-4fd3-9bd7-baf63cc5b12e None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:35.473071 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9124ba8-91fd-4fd3-9bd7-baf63cc5b12e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:35.483814 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9124ba8-91fd-4fd3-9bd7-baf63cc5b12e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:11:35.485347 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9124ba8-91fd-4fd3-9bd7-baf63cc5b12e None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:11:35.501391 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application [None req-f9124ba8-91fd-4fd3-9bd7-baf63cc5b12e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 12:11:35.501391 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:11:35.501391 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:11:35.501391 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:11:35.501391 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:11:35.501391 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:11:35.501391 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:11:35.501391 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:11:35.501391 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:11:35.501391 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:11:35.501391 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:11:35.501391 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:11:35.501391 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 12:11:35.501391 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 12:11:35.501391 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 12:11:35.501391 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 12:11:35.502355 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:11:35.502355 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:11:35.502355 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:11:35.502355 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:11:35.502355 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:11:35.502355 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:11:35.502355 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:11:35.502355 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application with Lock( Sep 15 12:11:35.502355 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:11:35.502355 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:11:35.502355 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:11:35.502355 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:11:35.502355 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:11:35.502355 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:11:35.502355 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:11:35.502355 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:11:35.502355 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 12:11:35.503289 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 12:11:35.503289 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 12:11:35.503289 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 15 12:11:35.503289 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 350/697] 10.222.0.15 () {56 vars in 1108 bytes} [Thu Sep 15 12:11:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:11:35.510001 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7864499-0ca9-40e7-b7c4-26231b8d2c91 None None] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:35.623790 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7864499-0ca9-40e7-b7c4-26231b8d2c91 None None] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:35.673813 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c010c89f-22f0-4f10-b635-62b79d9c6068 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:35.674425 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c010c89f-22f0-4f10-b635-62b79d9c6068 None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:35.674700 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c010c89f-22f0-4f10-b635-62b79d9c6068 None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:35.674966 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c010c89f-22f0-4f10-b635-62b79d9c6068 None admin] PATH_INFO: `/v3/domains` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:35.675603 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c010c89f-22f0-4f10-b635-62b79d9c6068 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81449) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:11:35.675901 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c010c89f-22f0-4f10-b635-62b79d9c6068 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:11:35.677893 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c010c89f-22f0-4f10-b635-62b79d9c6068 None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:11:35.696205 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 348/698] 10.222.0.15 () {56 vars in 1109 bytes} [Thu Sep 15 12:11:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:11:35.701602 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9124ba8-91fd-4fd3-9bd7-baf63cc5b12e None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:35.808973 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c010c89f-22f0-4f10-b635-62b79d9c6068 None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:35.816261 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:11:35.822827 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9124ba8-91fd-4fd3-9bd7-baf63cc5b12e None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:35.829222 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:11:35.901014 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66808c2d-4dec-4599-8313-c6ec09d40643 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:35.901716 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66808c2d-4dec-4599-8313-c6ec09d40643 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:35.901996 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66808c2d-4dec-4599-8313-c6ec09d40643 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:35.902263 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66808c2d-4dec-4599-8313-c6ec09d40643 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:35.913513 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66808c2d-4dec-4599-8313-c6ec09d40643 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:11:35.914942 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66808c2d-4dec-4599-8313-c6ec09d40643 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:11:35.934196 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application [None req-66808c2d-4dec-4599-8313-c6ec09d40643 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 15 12:11:35.934196 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:11:35.934196 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:11:35.934196 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:11:35.934196 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:11:35.934196 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:11:35.934196 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:11:35.934196 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:11:35.934196 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:11:35.934196 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:11:35.934196 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:11:35.934196 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:11:35.934196 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 15 12:11:35.934196 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 15 12:11:35.934196 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 15 12:11:35.934196 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 15 12:11:35.935634 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:11:35.935634 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:11:35.935634 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 15 12:11:35.935634 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 12:11:35.935634 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 15 12:11:35.935634 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 12:11:35.935634 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:11:35.935634 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:11:35.935634 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:11:35.935634 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:11:35.935634 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:11:35.935634 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application with Lock( Sep 15 12:11:35.935634 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:11:35.935634 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:11:35.935634 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:11:35.935634 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:11:35.935634 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:11:35.935634 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:11:35.936631 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:11:35.936631 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:11:35.936631 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 15 12:11:35.936631 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 15 12:11:35.936631 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 15 12:11:35.936631 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 15 12:11:35.936631 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 15 12:11:35.936631 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 15 12:11:35.936631 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 15 12:11:35.936631 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 15 12:11:35.936631 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 351/699] 10.222.0.15 () {56 vars in 1093 bytes} [Thu Sep 15 12:11:35 2022] GET /identity/v3/users/nova => generated 82 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:11:35.944548 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66808c2d-4dec-4599-8313-c6ec09d40643 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:35.998552 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c010c89f-22f0-4f10-b635-62b79d9c6068 None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:36.041597 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d12a7a-5e47-4fdf-8e7d-8ca7272945bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:36.042659 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d12a7a-5e47-4fdf-8e7d-8ca7272945bb None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:36.043158 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d12a7a-5e47-4fdf-8e7d-8ca7272945bb None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:36.043631 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d12a7a-5e47-4fdf-8e7d-8ca7272945bb None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:36.044414 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d12a7a-5e47-4fdf-8e7d-8ca7272945bb None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:11:36.046048 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d12a7a-5e47-4fdf-8e7d-8ca7272945bb None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:11:36.058609 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application [None req-96d12a7a-5e47-4fdf-8e7d-8ca7272945bb None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 15 12:11:36.058609 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:11:36.058609 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 15 12:11:36.058609 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 15 12:11:36.058609 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 15 12:11:36.058609 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 15 12:11:36.058609 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 15 12:11:36.058609 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 15 12:11:36.058609 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 15 12:11:36.058609 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application Sep 15 12:11:36.058609 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 15 12:11:36.058609 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application Sep 15 12:11:36.058609 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:11:36.058609 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:11:36.058609 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:11:36.058609 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:11:36.058609 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:11:36.059705 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:11:36.059705 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:11:36.059705 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:11:36.059705 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:11:36.059705 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:11:36.059705 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:11:36.059705 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 15 12:11:36.059705 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 15 12:11:36.059705 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 15 12:11:36.059705 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 15 12:11:36.059705 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:11:36.059705 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:11:36.059705 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:11:36.059705 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:11:36.059705 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:11:36.059705 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:11:36.060561 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:11:36.060561 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application with Lock( Sep 15 12:11:36.060561 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:11:36.060561 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:11:36.060561 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:11:36.060561 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:11:36.060561 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:11:36.060561 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:11:36.060561 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:11:36.060561 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:11:36.060561 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 15 12:11:36.060561 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 15 12:11:36.060561 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 15 12:11:36.060561 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application Sep 15 12:11:36.061801 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66808c2d-4dec-4599-8313-c6ec09d40643 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:36.063495 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 349/700] 10.222.0.15 () {56 vars in 1129 bytes} [Thu Sep 15 12:11:35 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:11:36.064968 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4a5e57-64a5-4a2b-9004-9346a927529c None None] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:36.065563 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4a5e57-64a5-4a2b-9004-9346a927529c None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:36.066044 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4a5e57-64a5-4a2b-9004-9346a927529c None None] PATH_INFO: `/` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:36.067780 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 350/701] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 12:11:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:11:36.068892 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4a5e57-64a5-4a2b-9004-9346a927529c None None] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:36.098794 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d157690-e934-4957-9951-579dcd8404ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:36.099336 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d157690-e934-4957-9951-579dcd8404ad None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:36.099612 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d157690-e934-4957-9951-579dcd8404ad None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:36.099879 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d157690-e934-4957-9951-579dcd8404ad None admin] PATH_INFO: `/v3/users` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:36.100781 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d157690-e934-4957-9951-579dcd8404ad None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:11:36.100893 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d157690-e934-4957-9951-579dcd8404ad None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:11:36.102305 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d157690-e934-4957-9951-579dcd8404ad None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:11:36.124936 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 352/702] 10.222.0.15 () {56 vars in 1133 bytes} [Thu Sep 15 12:11:35 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:11:36.126272 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825ddb85-586b-43b0-b357-d89bff0db1d3 None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:36.126555 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825ddb85-586b-43b0-b357-d89bff0db1d3 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:36.126814 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825ddb85-586b-43b0-b357-d89bff0db1d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:36.144524 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-825ddb85-586b-43b0-b357-d89bff0db1d3 None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:11:36.146294 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-825ddb85-586b-43b0-b357-d89bff0db1d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:11:36.156862 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:11:36.159792 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:11:36.187576 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4a5e57-64a5-4a2b-9004-9346a927529c None None] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:36.225203 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 353/703] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 12:11:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 12:11:36.226700 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825ddb85-586b-43b0-b357-d89bff0db1d3 None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:36.232372 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f584d1c4-f1e7-4ec8-9339-938c1f61e1ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:36.233432 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f584d1c4-f1e7-4ec8-9339-938c1f61e1ac None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:36.233891 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f584d1c4-f1e7-4ec8-9339-938c1f61e1ac None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:36.234327 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f584d1c4-f1e7-4ec8-9339-938c1f61e1ac None admin] PATH_INFO: `/v3/services` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:36.235180 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f584d1c4-f1e7-4ec8-9339-938c1f61e1ac None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81449) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:11:36.235659 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f584d1c4-f1e7-4ec8-9339-938c1f61e1ac None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:11:36.237275 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f584d1c4-f1e7-4ec8-9339-938c1f61e1ac None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:11:36.243301 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 351/704] 10.222.0.15 () {56 vars in 1124 bytes} [Thu Sep 15 12:11:36 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:11:36.251312 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f584d1c4-f1e7-4ec8-9339-938c1f61e1ac None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:36.399117 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825ddb85-586b-43b0-b357-d89bff0db1d3 None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:36.399301 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f584d1c4-f1e7-4ec8-9339-938c1f61e1ac None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:36.434337 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15ce166-dde1-46cc-a9fa-f770ddcd1156 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:36.435128 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15ce166-dde1-46cc-a9fa-f770ddcd1156 None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:36.435714 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15ce166-dde1-46cc-a9fa-f770ddcd1156 None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:36.436110 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15ce166-dde1-46cc-a9fa-f770ddcd1156 None admin] PATH_INFO: `/v3/services` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:36.436872 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15ce166-dde1-46cc-a9fa-f770ddcd1156 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81449) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:11:36.437174 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15ce166-dde1-46cc-a9fa-f770ddcd1156 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:11:36.438526 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15ce166-dde1-46cc-a9fa-f770ddcd1156 None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:11:36.440494 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce046f62-4a24-4e2b-b9d9-cda97a1d4ff0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:36.441236 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce046f62-4a24-4e2b-b9d9-cda97a1d4ff0 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:36.442028 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce046f62-4a24-4e2b-b9d9-cda97a1d4ff0 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:36.442395 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce046f62-4a24-4e2b-b9d9-cda97a1d4ff0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:36.444574 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 352/705] 10.222.0.15 () {56 vars in 1124 bytes} [Thu Sep 15 12:11:36 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:11:36.445755 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96475c1e-d95f-4a7c-8990-3c93115eeb6d None None] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:36.446086 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96475c1e-d95f-4a7c-8990-3c93115eeb6d None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:36.446352 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96475c1e-d95f-4a7c-8990-3c93115eeb6d None None] PATH_INFO: `/` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:36.447763 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 353/706] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 12:11:36 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:11:36.453648 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7173bcb1-b023-4f32-9486-96ae2edbf8a1 None None] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:36.453933 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7173bcb1-b023-4f32-9486-96ae2edbf8a1 None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:36.454491 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7173bcb1-b023-4f32-9486-96ae2edbf8a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:36.466113 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:11:36.479935 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.auth.core [None req-7173bcb1-b023-4f32-9486-96ae2edbf8a1 None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81449) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:11:36.485584 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-7173bcb1-b023-4f32-9486-96ae2edbf8a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:11:36.493152 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:11:36.585825 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce046f62-4a24-4e2b-b9d9-cda97a1d4ff0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:11:36.588074 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce046f62-4a24-4e2b-b9d9-cda97a1d4ff0 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:11:36.592824 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 354/707] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 12:11:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 15 12:11:36.594109 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7173bcb1-b023-4f32-9486-96ae2edbf8a1 None None] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:36.713659 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 354/708] 10.222.0.15 () {58 vars in 1282 bytes} [Thu Sep 15 12:11:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 488 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Sep 15 12:11:36.721895 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce046f62-4a24-4e2b-b9d9-cda97a1d4ff0 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:36.754354 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7173bcb1-b023-4f32-9486-96ae2edbf8a1 None None] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:36.777166 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:11:36.794709 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea688ab-78a4-4a8a-bb3f-b535fe56cc43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:36.795736 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea688ab-78a4-4a8a-bb3f-b535fe56cc43 None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:36.796224 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea688ab-78a4-4a8a-bb3f-b535fe56cc43 None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:36.796722 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea688ab-78a4-4a8a-bb3f-b535fe56cc43 None admin] PATH_INFO: `/v3/services/network` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:36.797575 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea688ab-78a4-4a8a-bb3f-b535fe56cc43 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:11:36.799371 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea688ab-78a4-4a8a-bb3f-b535fe56cc43 None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:11:36.801603 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:11:36.810228 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application [None req-9ea688ab-78a4-4a8a-bb3f-b535fe56cc43 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 15 12:11:36.810228 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:11:36.810228 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 15 12:11:36.810228 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 15 12:11:36.810228 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 15 12:11:36.810228 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 15 12:11:36.810228 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 15 12:11:36.810228 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 15 12:11:36.810228 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 15 12:11:36.810228 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application Sep 15 12:11:36.810228 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 15 12:11:36.810228 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application Sep 15 12:11:36.810228 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:11:36.810228 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:11:36.810228 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:11:36.810228 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:11:36.810228 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:11:36.811238 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:11:36.811238 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:11:36.811238 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:11:36.811238 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:11:36.811238 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:11:36.811238 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:11:36.811238 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 15 12:11:36.811238 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 15 12:11:36.811238 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 15 12:11:36.811238 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 15 12:11:36.811238 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:11:36.811238 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:11:36.811238 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:11:36.811238 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:11:36.811238 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:11:36.811238 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:11:36.812086 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:11:36.812086 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application with Lock( Sep 15 12:11:36.812086 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:11:36.812086 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:11:36.812086 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:11:36.812086 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:11:36.812086 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:11:36.812086 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:11:36.812086 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:11:36.812086 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:11:36.812086 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 15 12:11:36.812086 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 15 12:11:36.812086 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 15 12:11:36.812086 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application Sep 15 12:11:36.819931 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 355/709] 10.222.0.15 () {56 vars in 1111 bytes} [Thu Sep 15 12:11:36 2022] GET /identity/v3/services/network => generated 88 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:11:36.821458 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea688ab-78a4-4a8a-bb3f-b535fe56cc43 None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:36.835914 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce046f62-4a24-4e2b-b9d9-cda97a1d4ff0 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:36.901983 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50577649-28bd-48c7-80fe-d65fdb0387ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:36.902993 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50577649-28bd-48c7-80fe-d65fdb0387ce None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:36.903483 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50577649-28bd-48c7-80fe-d65fdb0387ce None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:36.903969 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50577649-28bd-48c7-80fe-d65fdb0387ce None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:36.923037 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50577649-28bd-48c7-80fe-d65fdb0387ce None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:11:36.924977 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50577649-28bd-48c7-80fe-d65fdb0387ce None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:11:36.952445 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application [None req-50577649-28bd-48c7-80fe-d65fdb0387ce None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 12:11:36.952445 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:11:36.952445 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:11:36.952445 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:11:36.952445 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:11:36.952445 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:11:36.952445 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:11:36.952445 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:11:36.952445 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:11:36.952445 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:11:36.952445 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:11:36.952445 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:11:36.952445 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 12:11:36.952445 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 12:11:36.952445 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 12:11:36.952445 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 12:11:36.953943 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:11:36.953943 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:11:36.953943 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:11:36.953943 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:11:36.953943 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:11:36.953943 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:11:36.953943 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:11:36.953943 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application with Lock( Sep 15 12:11:36.953943 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:11:36.953943 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:11:36.953943 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:11:36.953943 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:11:36.953943 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:11:36.953943 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:11:36.953943 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:11:36.953943 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:11:36.953943 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 12:11:36.954859 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 12:11:36.954859 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 12:11:36.954859 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 15 12:11:36.958250 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 355/710] 10.222.0.15 () {56 vars in 1108 bytes} [Thu Sep 15 12:11:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:11:36.959641 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50577649-28bd-48c7-80fe-d65fdb0387ce None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:36.996069 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea688ab-78a4-4a8a-bb3f-b535fe56cc43 None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:37.044309 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ac68e9-f90b-47c5-984a-b482b01abf8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:37.045211 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ac68e9-f90b-47c5-984a-b482b01abf8d None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:37.045926 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ac68e9-f90b-47c5-984a-b482b01abf8d None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:37.046355 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ac68e9-f90b-47c5-984a-b482b01abf8d None admin] PATH_INFO: `/v3/services` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:37.047740 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ac68e9-f90b-47c5-984a-b482b01abf8d None admin] RBAC: Adding query filter params (name=network) {{(pid=81449) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:11:37.048355 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ac68e9-f90b-47c5-984a-b482b01abf8d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:11:37.050266 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ac68e9-f90b-47c5-984a-b482b01abf8d None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:11:37.060981 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 356/711] 10.222.0.15 () {56 vars in 1112 bytes} [Thu Sep 15 12:11:36 2022] GET /identity/v3/services?name=network => generated 124 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:11:37.061905 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ac68e9-f90b-47c5-984a-b482b01abf8d None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:37.088367 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:11:37.093343 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50577649-28bd-48c7-80fe-d65fdb0387ce None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:37.136832 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec97f0b6-bda6-4283-8dec-df69fed3dddf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:37.137631 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec97f0b6-bda6-4283-8dec-df69fed3dddf None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:37.137980 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec97f0b6-bda6-4283-8dec-df69fed3dddf None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:37.138324 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec97f0b6-bda6-4283-8dec-df69fed3dddf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:37.151126 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:11:37.151285 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec97f0b6-bda6-4283-8dec-df69fed3dddf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:11:37.152933 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec97f0b6-bda6-4283-8dec-df69fed3dddf None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:11:37.169400 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application [None req-ec97f0b6-bda6-4283-8dec-df69fed3dddf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 12:11:37.169400 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:11:37.169400 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:11:37.169400 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:11:37.169400 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:11:37.169400 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:11:37.169400 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:11:37.169400 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:11:37.169400 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:11:37.169400 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:11:37.169400 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:11:37.169400 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:11:37.169400 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 12:11:37.169400 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 12:11:37.169400 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 12:11:37.169400 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 12:11:37.170417 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:11:37.170417 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:11:37.170417 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:11:37.170417 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:11:37.170417 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:11:37.170417 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:11:37.170417 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:11:37.170417 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application with Lock( Sep 15 12:11:37.170417 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:11:37.170417 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:11:37.170417 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:11:37.170417 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:11:37.170417 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:11:37.170417 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:11:37.170417 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:11:37.170417 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:11:37.170417 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 12:11:37.171749 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 12:11:37.171749 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 12:11:37.171749 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 15 12:11:37.171749 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 356/712] 10.222.0.15 () {56 vars in 1108 bytes} [Thu Sep 15 12:11:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:11:37.172527 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec97f0b6-bda6-4283-8dec-df69fed3dddf None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:37.207130 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ac68e9-f90b-47c5-984a-b482b01abf8d None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:37.253033 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3812ace1-14fd-4c71-9a0b-7e2c4e64fd63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:37.254124 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3812ace1-14fd-4c71-9a0b-7e2c4e64fd63 None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:37.255966 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3812ace1-14fd-4c71-9a0b-7e2c4e64fd63 None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:37.256419 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3812ace1-14fd-4c71-9a0b-7e2c4e64fd63 None admin] PATH_INFO: `/v3/domains` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:37.257238 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3812ace1-14fd-4c71-9a0b-7e2c4e64fd63 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81449) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:11:37.257693 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3812ace1-14fd-4c71-9a0b-7e2c4e64fd63 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:11:37.259222 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3812ace1-14fd-4c71-9a0b-7e2c4e64fd63 None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:11:37.275858 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 357/713] 10.222.0.15 () {56 vars in 1109 bytes} [Thu Sep 15 12:11:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:11:37.276999 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3812ace1-14fd-4c71-9a0b-7e2c4e64fd63 None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:37.294505 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec97f0b6-bda6-4283-8dec-df69fed3dddf None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:37.333165 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69029372-57ba-479c-8fe9-a21f14a9750b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:37.334358 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69029372-57ba-479c-8fe9-a21f14a9750b None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:37.334736 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69029372-57ba-479c-8fe9-a21f14a9750b None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:37.335102 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69029372-57ba-479c-8fe9-a21f14a9750b None admin] PATH_INFO: `/v3/services` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:37.336145 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69029372-57ba-479c-8fe9-a21f14a9750b None admin] RBAC: Adding query filter params (type=network) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:11:37.336827 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69029372-57ba-479c-8fe9-a21f14a9750b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:11:37.338370 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69029372-57ba-479c-8fe9-a21f14a9750b None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:11:37.348794 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 357/714] 10.222.0.15 () {56 vars in 1112 bytes} [Thu Sep 15 12:11:37 2022] GET /identity/v3/services?type=network => generated 124 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:11:37.348794 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69029372-57ba-479c-8fe9-a21f14a9750b None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:37.378276 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:11:37.468011 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3812ace1-14fd-4c71-9a0b-7e2c4e64fd63 None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:37.501925 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69029372-57ba-479c-8fe9-a21f14a9750b None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:37.529768 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:11:37.544338 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5542e9e6-fda2-4387-824a-281bc86d0ab4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:37.545151 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5542e9e6-fda2-4387-824a-281bc86d0ab4 None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:37.545877 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5542e9e6-fda2-4387-824a-281bc86d0ab4 None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:37.546250 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5542e9e6-fda2-4387-824a-281bc86d0ab4 None admin] PATH_INFO: `/v3/domains` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:37.547081 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5542e9e6-fda2-4387-824a-281bc86d0ab4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81449) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:11:37.547731 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5542e9e6-fda2-4387-824a-281bc86d0ab4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:11:37.549646 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5542e9e6-fda2-4387-824a-281bc86d0ab4 None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:11:37.564012 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 358/715] 10.222.0.15 () {56 vars in 1109 bytes} [Thu Sep 15 12:11:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:11:37.573834 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5542e9e6-fda2-4387-824a-281bc86d0ab4 None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:37.587037 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe2b0e1-8f4b-48fd-acd1-4f2492a18d55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:37.587702 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe2b0e1-8f4b-48fd-acd1-4f2492a18d55 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:37.587986 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe2b0e1-8f4b-48fd-acd1-4f2492a18d55 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:37.588251 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe2b0e1-8f4b-48fd-acd1-4f2492a18d55 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:37.595579 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe2b0e1-8f4b-48fd-acd1-4f2492a18d55 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:11:37.597088 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe2b0e1-8f4b-48fd-acd1-4f2492a18d55 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:11:37.611192 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application [None req-6fe2b0e1-8f4b-48fd-acd1-4f2492a18d55 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 12:11:37.611192 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:11:37.611192 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:11:37.611192 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:11:37.611192 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:11:37.611192 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:11:37.611192 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:11:37.611192 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:11:37.611192 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:11:37.611192 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:11:37.611192 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:11:37.611192 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:11:37.611192 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 15 12:11:37.611192 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 15 12:11:37.611192 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 15 12:11:37.612193 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 15 12:11:37.612193 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:11:37.612193 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:11:37.612193 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:11:37.612193 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:11:37.612193 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:11:37.612193 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:11:37.612193 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:11:37.612193 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application with Lock( Sep 15 12:11:37.612193 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:11:37.612193 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:11:37.612193 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:11:37.612193 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:11:37.612193 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:11:37.612193 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:11:37.612193 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:11:37.612193 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:11:37.613526 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 15 12:11:37.613526 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 15 12:11:37.613526 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 15 12:11:37.613526 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 15 12:11:37.613526 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 15 12:11:37.613526 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 15 12:11:37.613526 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 12:11:37.613526 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 15 12:11:37.613526 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 358/716] 10.222.0.15 () {56 vars in 1111 bytes} [Thu Sep 15 12:11:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:11:37.618004 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90a4788-c63f-477b-a62b-809b45166ff7 None None] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:37.618286 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90a4788-c63f-477b-a62b-809b45166ff7 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:37.618558 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90a4788-c63f-477b-a62b-809b45166ff7 None None] PATH_INFO: `/` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:37.620006 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 359/717] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 12:11:37 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:11:37.621732 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90a4788-c63f-477b-a62b-809b45166ff7 None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:37.699599 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5542e9e6-fda2-4387-824a-281bc86d0ab4 None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:37.716436 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:11:37.743667 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb21bec-1139-4353-ad12-33e3e3fb4c22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:37.744273 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb21bec-1139-4353-ad12-33e3e3fb4c22 None admin] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:37.744549 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb21bec-1139-4353-ad12-33e3e3fb4c22 None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:37.744841 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb21bec-1139-4353-ad12-33e3e3fb4c22 None admin] PATH_INFO: `/v3/users` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:37.745474 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb21bec-1139-4353-ad12-33e3e3fb4c22 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:11:37.746896 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bb21bec-1139-4353-ad12-33e3e3fb4c22 None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:11:37.747377 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90a4788-c63f-477b-a62b-809b45166ff7 None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:37.773460 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 359/718] 10.222.0.15 () {60 vars in 1132 bytes} [Thu Sep 15 12:11:37 2022] POST /identity/v3/users => generated 254 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 12:11:37.775080 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d872493b-fe9c-4d3f-8ff8-9b9d95ac9d93 None None] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:37.775347 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d872493b-fe9c-4d3f-8ff8-9b9d95ac9d93 None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:37.775664 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d872493b-fe9c-4d3f-8ff8-9b9d95ac9d93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:37.792197 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048608a5-8ccf-4e4f-8c70-c8c9c864f6c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:37.793359 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048608a5-8ccf-4e4f-8c70-c8c9c864f6c6 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:37.793846 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048608a5-8ccf-4e4f-8c70-c8c9c864f6c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:37.794378 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048608a5-8ccf-4e4f-8c70-c8c9c864f6c6 None admin] PATH_INFO: `/v3/projects` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:37.795478 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048608a5-8ccf-4e4f-8c70-c8c9c864f6c6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:11:37.796012 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048608a5-8ccf-4e4f-8c70-c8c9c864f6c6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:11:37.797823 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048608a5-8ccf-4e4f-8c70-c8c9c864f6c6 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:11:37.798886 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.auth.core [None req-d872493b-fe9c-4d3f-8ff8-9b9d95ac9d93 None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81449) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:11:37.800517 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-d872493b-fe9c-4d3f-8ff8-9b9d95ac9d93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:11:37.813617 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 360/719] 10.222.0.15 () {56 vars in 1148 bytes} [Thu Sep 15 12:11:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:11:37.820399 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048608a5-8ccf-4e4f-8c70-c8c9c864f6c6 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:37.843883 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:11:37.891194 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 360/720] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 12:11:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Sep 15 12:11:37.952906 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048608a5-8ccf-4e4f-8c70-c8c9c864f6c6 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:37.992732 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06287506-6155-4631-9912-5b6b27f31b22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:37.993404 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06287506-6155-4631-9912-5b6b27f31b22 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:37.993681 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06287506-6155-4631-9912-5b6b27f31b22 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:37.993948 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06287506-6155-4631-9912-5b6b27f31b22 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:37.994697 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06287506-6155-4631-9912-5b6b27f31b22 None admin] RBAC: Adding query filter params (role.id=9c769b5c82884766b2c510b1f0f1364c, scope.project.id=540a32ef88764e8c8d9201ae22f6ddaf, user.id=eface89e261840a2abb4206be26455c7) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:11:37.995005 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06287506-6155-4631-9912-5b6b27f31b22 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:11:37.996462 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06287506-6155-4631-9912-5b6b27f31b22 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:11:37.998862 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:11:38.010733 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 361/721] 10.222.0.15 () {56 vars in 1374 bytes} [Thu Sep 15 12:11:37 2022] GET /identity/v3/role_assignments?user.id=eface89e261840a2abb4206be26455c7&scope.project.id=540a32ef88764e8c8d9201ae22f6ddaf&role.id=9c769b5c82884766b2c510b1f0f1364c => generated 259 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:11:38.215904 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d872493b-fe9c-4d3f-8ff8-9b9d95ac9d93 None None] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:38.328629 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d872493b-fe9c-4d3f-8ff8-9b9d95ac9d93 None None] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:38.369822 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314ff1c4-ff74-4367-ba7b-afa80f40f737 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:38.370505 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314ff1c4-ff74-4367-ba7b-afa80f40f737 None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:38.370790 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314ff1c4-ff74-4367-ba7b-afa80f40f737 None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:38.371064 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314ff1c4-ff74-4367-ba7b-afa80f40f737 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:38.382011 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314ff1c4-ff74-4367-ba7b-afa80f40f737 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 12:11:38.386889 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application [None req-314ff1c4-ff74-4367-ba7b-afa80f40f737 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 15 12:11:38.386889 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:11:38.386889 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:11:38.386889 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:11:38.386889 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:11:38.386889 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:11:38.386889 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:11:38.386889 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:11:38.386889 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:11:38.386889 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:11:38.386889 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:11:38.386889 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:11:38.386889 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 12:11:38.386889 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 12:11:38.386889 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 12:11:38.386889 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 12:11:38.387808 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 12:11:38.387808 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 12:11:38.387808 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 12:11:38.387808 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 12:11:38.387808 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:11:38.387808 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:11:38.387808 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:11:38.387808 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:11:38.387808 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:11:38.387808 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:11:38.387808 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:11:38.387808 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application with Lock( Sep 15 12:11:38.387808 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:11:38.387808 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:11:38.387808 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:11:38.387808 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:11:38.388648 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:11:38.388648 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:11:38.388648 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:11:38.388648 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:11:38.388648 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 12:11:38.388648 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 12:11:38.388648 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 12:11:38.388648 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 12:11:38.388648 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 12:11:38.388648 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 12:11:38.388648 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 15 12:11:38.388648 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application Sep 15 12:11:38.388648 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 361/722] 10.222.0.15 () {56 vars in 1096 bytes} [Thu Sep 15 12:11:38 2022] GET /identity/v3/roles/admin => generated 83 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:11:38.394337 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06287506-6155-4631-9912-5b6b27f31b22 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:38.534086 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06287506-6155-4631-9912-5b6b27f31b22 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:38.571098 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4b7e40-e7b6-452c-94dc-d20c1cdf2e91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:38.571728 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4b7e40-e7b6-452c-94dc-d20c1cdf2e91 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:38.572004 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4b7e40-e7b6-452c-94dc-d20c1cdf2e91 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:38.572274 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4b7e40-e7b6-452c-94dc-d20c1cdf2e91 None admin] PATH_INFO: `/v3/roles` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:38.572979 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4b7e40-e7b6-452c-94dc-d20c1cdf2e91 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:11:38.573283 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4b7e40-e7b6-452c-94dc-d20c1cdf2e91 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:11:38.574672 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4b7e40-e7b6-452c-94dc-d20c1cdf2e91 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:11:38.584606 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 362/723] 10.222.0.15 () {56 vars in 1099 bytes} [Thu Sep 15 12:11:38 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:11:38.590351 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314ff1c4-ff74-4367-ba7b-afa80f40f737 None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:38.639067 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:11:38.737840 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314ff1c4-ff74-4367-ba7b-afa80f40f737 None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:38.789039 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab23283c-089d-43f5-9572-94bf5d0a540c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:38.789599 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab23283c-089d-43f5-9572-94bf5d0a540c None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:38.789877 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab23283c-089d-43f5-9572-94bf5d0a540c None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:38.790150 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab23283c-089d-43f5-9572-94bf5d0a540c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:38.808127 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab23283c-089d-43f5-9572-94bf5d0a540c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:11:38.809608 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab23283c-089d-43f5-9572-94bf5d0a540c None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:11:38.833297 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application [None req-ab23283c-089d-43f5-9572-94bf5d0a540c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 12:11:38.833297 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:11:38.833297 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:11:38.833297 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:11:38.833297 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:11:38.833297 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:11:38.833297 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:11:38.833297 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:11:38.833297 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:11:38.833297 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:11:38.833297 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:11:38.833297 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:11:38.833297 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 12:11:38.833297 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 12:11:38.833297 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 12:11:38.833297 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 12:11:38.834399 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:11:38.834399 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:11:38.834399 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:11:38.834399 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:11:38.834399 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:11:38.834399 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:11:38.834399 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:11:38.834399 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application with Lock( Sep 15 12:11:38.834399 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:11:38.834399 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:11:38.834399 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:11:38.834399 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:11:38.834399 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:11:38.834399 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:11:38.834399 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:11:38.834399 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:11:38.834399 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 12:11:38.835316 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 12:11:38.835316 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 12:11:38.835316 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application Sep 15 12:11:38.835316 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 362/724] 10.222.0.15 () {56 vars in 1108 bytes} [Thu Sep 15 12:11:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:11:38.841473 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4b7e40-e7b6-452c-94dc-d20c1cdf2e91 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:38.937252 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:11:38.972770 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4b7e40-e7b6-452c-94dc-d20c1cdf2e91 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:39.029984 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4070deb-8b38-450b-9e1c-a65cf061dbc5 None None] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:39.030219 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8f3f33-da70-45e8-99c8-9e2d120780eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:39.030471 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4070deb-8b38-450b-9e1c-a65cf061dbc5 None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:39.030846 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8f3f33-da70-45e8-99c8-9e2d120780eb None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:39.030954 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4070deb-8b38-450b-9e1c-a65cf061dbc5 None None] PATH_INFO: `/` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:39.031123 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8f3f33-da70-45e8-99c8-9e2d120780eb None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:39.031396 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8f3f33-da70-45e8-99c8-9e2d120780eb None admin] PATH_INFO: `/v3/domains` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:39.032038 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8f3f33-da70-45e8-99c8-9e2d120780eb None admin] RBAC: Adding query filter params (name=Default) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:11:39.032337 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8f3f33-da70-45e8-99c8-9e2d120780eb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:11:39.032436 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 363/725] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 12:11:39 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:11:39.033830 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8f3f33-da70-45e8-99c8-9e2d120780eb None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:11:39.038043 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1b0b3e-04ee-4876-b44c-fc92eeb9c12f None None] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:39.038330 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1b0b3e-04ee-4876-b44c-fc92eeb9c12f None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:39.038607 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1b0b3e-04ee-4876-b44c-fc92eeb9c12f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:39.047613 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 363/726] 10.222.0.15 () {56 vars in 1109 bytes} [Thu Sep 15 12:11:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:11:39.053735 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8f3f33-da70-45e8-99c8-9e2d120780eb None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:39.079720 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.auth.core [None req-6b1b0b3e-04ee-4876-b44c-fc92eeb9c12f None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81449) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:11:39.080946 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-6b1b0b3e-04ee-4876-b44c-fc92eeb9c12f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:11:39.238640 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8f3f33-da70-45e8-99c8-9e2d120780eb None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:39.242390 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 364/727] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 12:11:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 15 12:11:39.268605 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:11:39.291544 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c536098c-bae2-47be-a2d4-b966013355be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:39.292627 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c536098c-bae2-47be-a2d4-b966013355be None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:39.293042 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c536098c-bae2-47be-a2d4-b966013355be None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:39.293319 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c536098c-bae2-47be-a2d4-b966013355be None admin] PATH_INFO: `/v3/users/swift` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:39.301023 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c536098c-bae2-47be-a2d4-b966013355be None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:11:39.302968 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c536098c-bae2-47be-a2d4-b966013355be None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:11:39.314920 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application [None req-c536098c-bae2-47be-a2d4-b966013355be None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 15 12:11:39.314920 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:11:39.314920 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:11:39.314920 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:11:39.314920 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:11:39.314920 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:11:39.314920 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:11:39.314920 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:11:39.314920 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:11:39.314920 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:11:39.314920 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:11:39.314920 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:11:39.314920 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 15 12:11:39.314920 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 15 12:11:39.314920 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 15 12:11:39.314920 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 15 12:11:39.317642 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:11:39.317642 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:11:39.317642 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 15 12:11:39.317642 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 12:11:39.317642 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 15 12:11:39.317642 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 12:11:39.317642 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:11:39.317642 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:11:39.317642 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:11:39.317642 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:11:39.317642 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:11:39.317642 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application with Lock( Sep 15 12:11:39.317642 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:11:39.317642 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:11:39.317642 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:11:39.317642 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:11:39.317642 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:11:39.317642 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:11:39.319862 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:11:39.319862 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:11:39.319862 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 15 12:11:39.319862 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 15 12:11:39.319862 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 15 12:11:39.319862 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 15 12:11:39.319862 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 15 12:11:39.319862 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 15 12:11:39.319862 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 15 12:11:39.319862 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 15 12:11:39.319862 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 364/728] 10.222.0.15 () {56 vars in 1096 bytes} [Thu Sep 15 12:11:39 2022] GET /identity/v3/users/swift => generated 83 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:11:39.325868 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1b0b3e-04ee-4876-b44c-fc92eeb9c12f None None] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:39.342439 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:11:39.487401 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1b0b3e-04ee-4876-b44c-fc92eeb9c12f None None] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:39.537443 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8802848-a1b7-4c91-9464-dae7b2c2aa16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:39.538058 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8802848-a1b7-4c91-9464-dae7b2c2aa16 None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:39.538337 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8802848-a1b7-4c91-9464-dae7b2c2aa16 None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:39.538608 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8802848-a1b7-4c91-9464-dae7b2c2aa16 None admin] PATH_INFO: `/v3/users` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:39.539340 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8802848-a1b7-4c91-9464-dae7b2c2aa16 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81449) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:11:39.539644 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8802848-a1b7-4c91-9464-dae7b2c2aa16 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:11:39.541097 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8802848-a1b7-4c91-9464-dae7b2c2aa16 None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:11:39.558798 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c536098c-bae2-47be-a2d4-b966013355be None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:39.568791 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 365/729] 10.222.0.15 () {56 vars in 1135 bytes} [Thu Sep 15 12:11:39 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:11:39.568791 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8802848-a1b7-4c91-9464-dae7b2c2aa16 None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:39.694019 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:11:39.699448 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c536098c-bae2-47be-a2d4-b966013355be None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:39.730474 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8802848-a1b7-4c91-9464-dae7b2c2aa16 None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:39.749109 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1b5165-062b-42f5-bebb-5a8894ee7938 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:39.749740 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1b5165-062b-42f5-bebb-5a8894ee7938 None admin] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:39.750021 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1b5165-062b-42f5-bebb-5a8894ee7938 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:39.750294 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1b5165-062b-42f5-bebb-5a8894ee7938 None admin] PATH_INFO: `/v3/services` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:39.750965 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1b5165-062b-42f5-bebb-5a8894ee7938 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:11:39.752427 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1b5165-062b-42f5-bebb-5a8894ee7938 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:11:39.773314 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684d7ae8-b2ce-41c6-9ed6-d27b8d74465c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:39.774057 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684d7ae8-b2ce-41c6-9ed6-d27b8d74465c None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:39.774337 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684d7ae8-b2ce-41c6-9ed6-d27b8d74465c None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:39.774757 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684d7ae8-b2ce-41c6-9ed6-d27b8d74465c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:39.784580 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 365/730] 10.222.0.15 () {60 vars in 1141 bytes} [Thu Sep 15 12:11:39 2022] POST /identity/v3/services => generated 251 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 15 12:11:39.859905 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684d7ae8-b2ce-41c6-9ed6-d27b8d74465c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:11:39.861338 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684d7ae8-b2ce-41c6-9ed6-d27b8d74465c None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:11:39.938339 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:11:39.978846 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 366/731] 10.222.0.15 () {58 vars in 1282 bytes} [Thu Sep 15 12:11:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 406 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Sep 15 12:11:39.980465 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1b5165-062b-42f5-bebb-5a8894ee7938 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:40.028957 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:11:40.104929 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1b5165-062b-42f5-bebb-5a8894ee7938 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:40.142992 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656ef239-d261-4117-b280-59beec48b7dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:40.143584 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656ef239-d261-4117-b280-59beec48b7dc None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:40.143871 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656ef239-d261-4117-b280-59beec48b7dc None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:40.144144 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656ef239-d261-4117-b280-59beec48b7dc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:40.155410 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656ef239-d261-4117-b280-59beec48b7dc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:11:40.156901 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656ef239-d261-4117-b280-59beec48b7dc None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:11:40.173460 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application [None req-656ef239-d261-4117-b280-59beec48b7dc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 12:11:40.173460 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:11:40.173460 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:11:40.173460 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:11:40.173460 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:11:40.173460 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:11:40.173460 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:11:40.173460 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:11:40.173460 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:11:40.173460 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:11:40.173460 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:11:40.173460 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:11:40.173460 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 12:11:40.173460 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 12:11:40.173460 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 12:11:40.173460 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 12:11:40.174473 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:11:40.174473 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:11:40.174473 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:11:40.174473 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:11:40.174473 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:11:40.174473 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:11:40.174473 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:11:40.174473 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application with Lock( Sep 15 12:11:40.174473 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:11:40.174473 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:11:40.174473 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:11:40.174473 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:11:40.174473 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:11:40.174473 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:11:40.174473 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:11:40.174473 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:11:40.174473 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 12:11:40.175376 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 12:11:40.175376 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 12:11:40.175376 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 15 12:11:40.180052 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 366/732] 10.222.0.15 () {56 vars in 1108 bytes} [Thu Sep 15 12:11:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:11:40.181526 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684d7ae8-b2ce-41c6-9ed6-d27b8d74465c None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:40.286949 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5710bb-9bb0-40e9-b5fd-ec0c4541b485 None None] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:40.287662 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5710bb-9bb0-40e9-b5fd-ec0c4541b485 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:40.288165 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5710bb-9bb0-40e9-b5fd-ec0c4541b485 None None] PATH_INFO: `/` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:40.290638 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 367/733] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 12:11:40 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 15 12:11:40.297140 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02a0c7b-5ca1-46fd-b15a-e5cdd295e71f None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:40.297140 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02a0c7b-5ca1-46fd-b15a-e5cdd295e71f None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:40.297593 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02a0c7b-5ca1-46fd-b15a-e5cdd295e71f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:40.326727 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-f02a0c7b-5ca1-46fd-b15a-e5cdd295e71f None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:11:40.327919 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-f02a0c7b-5ca1-46fd-b15a-e5cdd295e71f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:11:40.343359 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684d7ae8-b2ce-41c6-9ed6-d27b8d74465c None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:40.415956 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d16ab3-8ae6-42dd-9955-9a59980698e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:40.416856 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d16ab3-8ae6-42dd-9955-9a59980698e7 None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:40.417294 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d16ab3-8ae6-42dd-9955-9a59980698e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:40.417707 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d16ab3-8ae6-42dd-9955-9a59980698e7 None admin] PATH_INFO: `/v3/domains` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:40.418485 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d16ab3-8ae6-42dd-9955-9a59980698e7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81449) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:11:40.418939 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d16ab3-8ae6-42dd-9955-9a59980698e7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:11:40.420433 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d16ab3-8ae6-42dd-9955-9a59980698e7 None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:11:40.428463 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 368/734] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 12:11:40 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 15 12:11:40.429882 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036bfb23-fc49-42a7-8a56-160f9e75408d None None] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:40.430368 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036bfb23-fc49-42a7-8a56-160f9e75408d None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:40.430815 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036bfb23-fc49-42a7-8a56-160f9e75408d None None] PATH_INFO: `/` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:40.432228 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 369/735] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 12:11:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:11:40.437907 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 367/736] 10.222.0.15 () {56 vars in 1109 bytes} [Thu Sep 15 12:11:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:11:40.443458 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036bfb23-fc49-42a7-8a56-160f9e75408d None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:40.458642 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5635eda5-c2b6-498d-8980-f883aa3628d2 None None] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:40.459189 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5635eda5-c2b6-498d-8980-f883aa3628d2 None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:40.459666 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5635eda5-c2b6-498d-8980-f883aa3628d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:40.480149 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:11:40.490733 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.auth.core [None req-5635eda5-c2b6-498d-8980-f883aa3628d2 None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81449) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:11:40.493450 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-5635eda5-c2b6-498d-8980-f883aa3628d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:11:40.538480 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:11:40.618836 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036bfb23-fc49-42a7-8a56-160f9e75408d None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:40.631018 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 368/737] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 12:11:40 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Sep 15 12:11:40.680205 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35933df8-062b-4607-af9f-cea0b658dd8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:40.680794 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35933df8-062b-4607-af9f-cea0b658dd8c None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:40.681069 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35933df8-062b-4607-af9f-cea0b658dd8c None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:40.681334 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35933df8-062b-4607-af9f-cea0b658dd8c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:40.694420 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35933df8-062b-4607-af9f-cea0b658dd8c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:11:40.695850 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35933df8-062b-4607-af9f-cea0b658dd8c None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:11:40.710714 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application [None req-35933df8-062b-4607-af9f-cea0b658dd8c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 12:11:40.710714 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:11:40.710714 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:11:40.710714 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:11:40.710714 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:11:40.710714 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:11:40.710714 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:11:40.710714 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:11:40.710714 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:11:40.710714 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:11:40.710714 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:11:40.710714 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:11:40.710714 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 15 12:11:40.710714 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 15 12:11:40.710714 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 15 12:11:40.711597 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 15 12:11:40.711597 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:11:40.711597 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:11:40.711597 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:11:40.711597 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:11:40.711597 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:11:40.711597 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:11:40.711597 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:11:40.711597 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application with Lock( Sep 15 12:11:40.711597 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:11:40.711597 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:11:40.711597 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:11:40.711597 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:11:40.711597 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:11:40.711597 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:11:40.711597 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:11:40.711597 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:11:40.712899 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 15 12:11:40.712899 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 15 12:11:40.712899 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 15 12:11:40.712899 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 15 12:11:40.712899 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 15 12:11:40.712899 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 15 12:11:40.712899 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 12:11:40.712899 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 15 12:11:40.712899 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 370/738] 10.222.0.15 () {56 vars in 1111 bytes} [Thu Sep 15 12:11:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:11:40.717675 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5635eda5-c2b6-498d-8980-f883aa3628d2 None None] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:40.763528 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35933df8-062b-4607-af9f-cea0b658dd8c None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:40.843991 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5635eda5-c2b6-498d-8980-f883aa3628d2 None None] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:40.851052 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:11:40.875470 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35933df8-062b-4607-af9f-cea0b658dd8c None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:40.908253 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f595e53c-a13b-4fd0-90c4-f39740c0863f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:40.909340 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f595e53c-a13b-4fd0-90c4-f39740c0863f None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:40.909858 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f595e53c-a13b-4fd0-90c4-f39740c0863f None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:40.910336 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f595e53c-a13b-4fd0-90c4-f39740c0863f None admin] PATH_INFO: `/v3/projects` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:40.911354 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f595e53c-a13b-4fd0-90c4-f39740c0863f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81449) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:11:40.911889 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f595e53c-a13b-4fd0-90c4-f39740c0863f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:11:40.913620 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f595e53c-a13b-4fd0-90c4-f39740c0863f None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:11:40.941053 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a509146c-2a9b-42d7-a621-374522ed4132 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:40.942653 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a509146c-2a9b-42d7-a621-374522ed4132 None admin] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:40.945457 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a509146c-2a9b-42d7-a621-374522ed4132 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:40.946113 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a509146c-2a9b-42d7-a621-374522ed4132 None admin] PATH_INFO: `/v3/services` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:40.946903 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 369/739] 10.222.0.15 () {56 vars in 1148 bytes} [Thu Sep 15 12:11:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:11:40.947627 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a509146c-2a9b-42d7-a621-374522ed4132 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:11:40.949247 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a509146c-2a9b-42d7-a621-374522ed4132 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:11:40.953063 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f595e53c-a13b-4fd0-90c4-f39740c0863f None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:40.977637 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:11:41.002167 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 371/740] 10.222.0.15 () {60 vars in 1141 bytes} [Thu Sep 15 12:11:40 2022] POST /identity/v3/services => generated 240 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 15 12:11:41.002167 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a509146c-2a9b-42d7-a621-374522ed4132 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:41.100611 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f595e53c-a13b-4fd0-90c4-f39740c0863f None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:41.147427 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a509146c-2a9b-42d7-a621-374522ed4132 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:41.152269 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed0f9fb-e96b-43a5-bb5c-df296bbd00c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:41.153284 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed0f9fb-e96b-43a5-bb5c-df296bbd00c9 None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:41.153799 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed0f9fb-e96b-43a5-bb5c-df296bbd00c9 None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:41.154283 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed0f9fb-e96b-43a5-bb5c-df296bbd00c9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:41.155278 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed0f9fb-e96b-43a5-bb5c-df296bbd00c9 None admin] RBAC: Adding query filter params (role.id=9c769b5c82884766b2c510b1f0f1364c, scope.project.id=540a32ef88764e8c8d9201ae22f6ddaf, user.id=efa0d3dc5b7540d59fdd59bbf64a04b6) {{(pid=81449) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:11:41.156336 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed0f9fb-e96b-43a5-bb5c-df296bbd00c9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:11:41.158460 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed0f9fb-e96b-43a5-bb5c-df296bbd00c9 None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:11:41.167579 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 370/741] 10.222.0.15 () {56 vars in 1374 bytes} [Thu Sep 15 12:11:40 2022] GET /identity/v3/role_assignments?user.id=efa0d3dc5b7540d59fdd59bbf64a04b6&scope.project.id=540a32ef88764e8c8d9201ae22f6ddaf&role.id=9c769b5c82884766b2c510b1f0f1364c => generated 259 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:11:41.169300 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c286f5-8a9d-4418-89f4-a1dfc8bf0563 None None] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:41.169783 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c286f5-8a9d-4418-89f4-a1dfc8bf0563 None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:41.170238 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c286f5-8a9d-4418-89f4-a1dfc8bf0563 None None] PATH_INFO: `/` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:41.172774 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 371/742] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 12:11:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:11:41.183773 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13509c2-428d-4099-9ef6-83fe595af20f None None] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:41.183773 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13509c2-428d-4099-9ef6-83fe595af20f None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:41.183773 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13509c2-428d-4099-9ef6-83fe595af20f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:41.198171 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b803d02a-42d7-4335-85d4-2d3483249f5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:41.198892 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b803d02a-42d7-4335-85d4-2d3483249f5d None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:41.199272 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b803d02a-42d7-4335-85d4-2d3483249f5d None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:41.199642 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b803d02a-42d7-4335-85d4-2d3483249f5d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:41.202104 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:11:41.205289 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.auth.core [None req-b13509c2-428d-4099-9ef6-83fe595af20f None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81449) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:11:41.206772 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-b13509c2-428d-4099-9ef6-83fe595af20f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:11:41.216008 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b803d02a-42d7-4335-85d4-2d3483249f5d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 12:11:41.222127 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application [None req-b803d02a-42d7-4335-85d4-2d3483249f5d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 15 12:11:41.222127 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:11:41.222127 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:11:41.222127 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:11:41.222127 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:11:41.222127 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:11:41.222127 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:11:41.222127 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:11:41.222127 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:11:41.222127 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:11:41.222127 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:11:41.222127 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:11:41.222127 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 12:11:41.222127 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 12:11:41.222127 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 12:11:41.222127 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 12:11:41.223888 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 12:11:41.223888 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 12:11:41.223888 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 12:11:41.223888 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 12:11:41.223888 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:11:41.223888 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:11:41.223888 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:11:41.223888 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:11:41.223888 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:11:41.223888 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:11:41.223888 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:11:41.223888 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application with Lock( Sep 15 12:11:41.223888 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:11:41.223888 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:11:41.223888 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:11:41.223888 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:11:41.225001 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:11:41.225001 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:11:41.225001 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:11:41.225001 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:11:41.225001 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 12:11:41.225001 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 12:11:41.225001 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 12:11:41.225001 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 12:11:41.225001 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 12:11:41.225001 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 12:11:41.225001 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 15 12:11:41.225001 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 15 12:11:41.226762 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 372/743] 10.222.0.15 () {56 vars in 1096 bytes} [Thu Sep 15 12:11:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:11:41.233803 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b803d02a-42d7-4335-85d4-2d3483249f5d None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:41.314817 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 372/744] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 12:11:41 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 15 12:11:41.361959 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b803d02a-42d7-4335-85d4-2d3483249f5d None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:41.399069 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3edd21f-1612-4b9a-a552-6b99aaefcd12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:41.400045 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3edd21f-1612-4b9a-a552-6b99aaefcd12 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:41.400514 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3edd21f-1612-4b9a-a552-6b99aaefcd12 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:41.401025 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3edd21f-1612-4b9a-a552-6b99aaefcd12 None admin] PATH_INFO: `/v3/roles` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:41.401864 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3edd21f-1612-4b9a-a552-6b99aaefcd12 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:11:41.402346 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3edd21f-1612-4b9a-a552-6b99aaefcd12 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:11:41.404040 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3edd21f-1612-4b9a-a552-6b99aaefcd12 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:11:41.414320 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 373/745] 10.222.0.15 () {56 vars in 1099 bytes} [Thu Sep 15 12:11:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 12:11:41.419614 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13509c2-428d-4099-9ef6-83fe595af20f None None] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:41.445195 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:11:41.575279 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13509c2-428d-4099-9ef6-83fe595af20f None None] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:41.612130 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad2a811-e2a4-49d3-84f6-2cf04a83f634 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:41.612785 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad2a811-e2a4-49d3-84f6-2cf04a83f634 None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:41.613142 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad2a811-e2a4-49d3-84f6-2cf04a83f634 None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:41.613411 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad2a811-e2a4-49d3-84f6-2cf04a83f634 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:41.623927 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad2a811-e2a4-49d3-84f6-2cf04a83f634 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:11:41.625412 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad2a811-e2a4-49d3-84f6-2cf04a83f634 None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:11:41.640196 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application [None req-0ad2a811-e2a4-49d3-84f6-2cf04a83f634 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 12:11:41.640196 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:11:41.640196 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:11:41.640196 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:11:41.640196 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:11:41.640196 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:11:41.640196 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:11:41.640196 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:11:41.640196 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:11:41.640196 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:11:41.640196 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:11:41.640196 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:11:41.640196 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 12:11:41.640196 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 12:11:41.640196 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 12:11:41.640196 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 12:11:41.641223 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:11:41.641223 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:11:41.641223 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:11:41.641223 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:11:41.641223 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:11:41.641223 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:11:41.641223 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:11:41.641223 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application with Lock( Sep 15 12:11:41.641223 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:11:41.641223 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:11:41.641223 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:11:41.641223 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:11:41.641223 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:11:41.641223 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:11:41.641223 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:11:41.641223 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:11:41.641223 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 12:11:41.642132 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 12:11:41.642132 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 12:11:41.642132 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application Sep 15 12:11:41.642132 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 373/746] 10.222.0.15 () {56 vars in 1108 bytes} [Thu Sep 15 12:11:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:11:41.647700 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3edd21f-1612-4b9a-a552-6b99aaefcd12 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:41.673743 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad2a811-e2a4-49d3-84f6-2cf04a83f634 None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:41.741129 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:11:41.766073 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:11:41.799001 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad2a811-e2a4-49d3-84f6-2cf04a83f634 None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:41.836983 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aaeeaf4-67a2-49f0-b122-0fac0973e5fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:41.837553 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aaeeaf4-67a2-49f0-b122-0fac0973e5fd None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:41.837831 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aaeeaf4-67a2-49f0-b122-0fac0973e5fd None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:41.838101 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aaeeaf4-67a2-49f0-b122-0fac0973e5fd None admin] PATH_INFO: `/v3/roles/service` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:41.849871 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aaeeaf4-67a2-49f0-b122-0fac0973e5fd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 12:11:41.856425 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application [None req-1aaeeaf4-67a2-49f0-b122-0fac0973e5fd None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 15 12:11:41.856425 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:11:41.856425 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:11:41.856425 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:11:41.856425 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:11:41.856425 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:11:41.856425 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:11:41.856425 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:11:41.856425 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:11:41.856425 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:11:41.856425 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:11:41.856425 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:11:41.856425 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 12:11:41.856425 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 12:11:41.856425 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 12:11:41.856425 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 12:11:41.857416 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 12:11:41.857416 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 12:11:41.857416 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 12:11:41.857416 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 12:11:41.857416 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:11:41.857416 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:11:41.857416 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:11:41.857416 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:11:41.857416 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:11:41.857416 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:11:41.857416 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:11:41.857416 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application with Lock( Sep 15 12:11:41.857416 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:11:41.857416 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:11:41.857416 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:11:41.857416 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:11:41.858266 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:11:41.858266 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:11:41.858266 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:11:41.858266 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:11:41.858266 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 12:11:41.858266 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 12:11:41.858266 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 12:11:41.858266 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 12:11:41.858266 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 12:11:41.858266 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 12:11:41.858266 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 15 12:11:41.858266 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application Sep 15 12:11:41.858266 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 374/747] 10.222.0.15 () {56 vars in 1102 bytes} [Thu Sep 15 12:11:41 2022] GET /identity/v3/roles/service => generated 85 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 15 12:11:41.864977 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aaeeaf4-67a2-49f0-b122-0fac0973e5fd None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:41.876657 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3edd21f-1612-4b9a-a552-6b99aaefcd12 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:41.960700 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02b5a5a-e968-4f25-bc21-23e40ce4cb0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:41.961503 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02b5a5a-e968-4f25-bc21-23e40ce4cb0c None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:41.961788 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02b5a5a-e968-4f25-bc21-23e40ce4cb0c None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:41.962054 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02b5a5a-e968-4f25-bc21-23e40ce4cb0c None admin] PATH_INFO: `/v3/domains` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:41.962920 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02b5a5a-e968-4f25-bc21-23e40ce4cb0c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:11:41.963221 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02b5a5a-e968-4f25-bc21-23e40ce4cb0c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:11:41.964591 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02b5a5a-e968-4f25-bc21-23e40ce4cb0c None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:11:41.992904 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 374/748] 10.222.0.15 () {56 vars in 1109 bytes} [Thu Sep 15 12:11:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 344 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:11:41.999795 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02b5a5a-e968-4f25-bc21-23e40ce4cb0c None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:42.005189 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aaeeaf4-67a2-49f0-b122-0fac0973e5fd None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:42.061218 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:11:42.075878 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790964cb-197a-4526-9c9f-0aa275dc3741 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:42.076506 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790964cb-197a-4526-9c9f-0aa275dc3741 None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:42.076806 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790964cb-197a-4526-9c9f-0aa275dc3741 None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:42.077083 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790964cb-197a-4526-9c9f-0aa275dc3741 None admin] PATH_INFO: `/v3/roles` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:42.077754 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790964cb-197a-4526-9c9f-0aa275dc3741 None admin] RBAC: Adding query filter params (name=service) {{(pid=81449) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:11:42.078064 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790964cb-197a-4526-9c9f-0aa275dc3741 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:11:42.079442 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790964cb-197a-4526-9c9f-0aa275dc3741 None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:11:42.098964 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 375/749] 10.222.0.15 () {56 vars in 1103 bytes} [Thu Sep 15 12:11:41 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:11:42.104241 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790964cb-197a-4526-9c9f-0aa275dc3741 None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:42.187524 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02b5a5a-e968-4f25-bc21-23e40ce4cb0c None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:42.220977 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:11:42.236087 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790964cb-197a-4526-9c9f-0aa275dc3741 None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:42.258824 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04affaed-c9ff-48c3-8af9-fa497d46afa4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:42.259489 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04affaed-c9ff-48c3-8af9-fa497d46afa4 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:42.259769 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04affaed-c9ff-48c3-8af9-fa497d46afa4 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:42.260038 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04affaed-c9ff-48c3-8af9-fa497d46afa4 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:42.275919 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04affaed-c9ff-48c3-8af9-fa497d46afa4 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:11:42.277410 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04affaed-c9ff-48c3-8af9-fa497d46afa4 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:11:42.286776 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f39faa-8298-4351-bda9-00a749e3cad2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:42.287422 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f39faa-8298-4351-bda9-00a749e3cad2 None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:42.287697 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f39faa-8298-4351-bda9-00a749e3cad2 None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:42.287961 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f39faa-8298-4351-bda9-00a749e3cad2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:42.302499 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application [None req-04affaed-c9ff-48c3-8af9-fa497d46afa4 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 15 12:11:42.302499 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:11:42.302499 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:11:42.302499 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:11:42.302499 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:11:42.302499 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:11:42.302499 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:11:42.302499 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:11:42.302499 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:11:42.302499 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:11:42.302499 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:11:42.302499 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:11:42.302499 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 15 12:11:42.302499 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 15 12:11:42.302499 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 15 12:11:42.302499 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 15 12:11:42.303691 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:11:42.303691 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:11:42.303691 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 15 12:11:42.303691 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 12:11:42.303691 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 15 12:11:42.303691 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 12:11:42.303691 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:11:42.303691 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:11:42.303691 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:11:42.303691 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:11:42.303691 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:11:42.303691 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application with Lock( Sep 15 12:11:42.303691 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:11:42.303691 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:11:42.303691 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:11:42.303691 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:11:42.303691 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:11:42.303691 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:11:42.304855 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:11:42.304855 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:11:42.304855 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 15 12:11:42.304855 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 15 12:11:42.304855 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 15 12:11:42.304855 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 15 12:11:42.304855 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 15 12:11:42.304855 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 15 12:11:42.304855 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 15 12:11:42.304855 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 15 12:11:42.308013 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 375/750] 10.222.0.15 () {56 vars in 1093 bytes} [Thu Sep 15 12:11:41 2022] GET /identity/v3/users/nova => generated 82 bytes in 308 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:11:42.315152 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04affaed-c9ff-48c3-8af9-fa497d46afa4 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:42.318189 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f39faa-8298-4351-bda9-00a749e3cad2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:11:42.319798 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f39faa-8298-4351-bda9-00a749e3cad2 None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:11:42.352825 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application [None req-f3f39faa-8298-4351-bda9-00a749e3cad2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 12:11:42.352825 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:11:42.352825 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:11:42.352825 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:11:42.352825 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:11:42.352825 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:11:42.352825 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:11:42.352825 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:11:42.352825 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:11:42.352825 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:11:42.352825 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:11:42.352825 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:11:42.352825 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 12:11:42.352825 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 12:11:42.352825 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 12:11:42.352825 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 12:11:42.353982 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:11:42.353982 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:11:42.353982 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:11:42.353982 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:11:42.353982 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:11:42.353982 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:11:42.353982 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:11:42.353982 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application with Lock( Sep 15 12:11:42.353982 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:11:42.353982 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:11:42.353982 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:11:42.353982 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:11:42.353982 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:11:42.353982 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:11:42.353982 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:11:42.353982 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:11:42.353982 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 12:11:42.355359 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 12:11:42.355359 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 12:11:42.355359 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application Sep 15 12:11:42.355359 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 376/751] 10.222.0.15 () {56 vars in 1108 bytes} [Thu Sep 15 12:11:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 15 12:11:42.362647 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f39faa-8298-4351-bda9-00a749e3cad2 None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:42.443107 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:11:42.485783 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04affaed-c9ff-48c3-8af9-fa497d46afa4 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:42.523187 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602e88b8-5fc1-44a0-8cf8-583f7e7f9eca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:42.523888 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602e88b8-5fc1-44a0-8cf8-583f7e7f9eca None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:42.524234 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602e88b8-5fc1-44a0-8cf8-583f7e7f9eca None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:42.524855 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602e88b8-5fc1-44a0-8cf8-583f7e7f9eca None admin] PATH_INFO: `/v3/users` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:42.525401 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602e88b8-5fc1-44a0-8cf8-583f7e7f9eca None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:11:42.525767 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602e88b8-5fc1-44a0-8cf8-583f7e7f9eca None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:11:42.527232 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602e88b8-5fc1-44a0-8cf8-583f7e7f9eca None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:11:42.544526 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 376/752] 10.222.0.15 () {56 vars in 1133 bytes} [Thu Sep 15 12:11:42 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:11:42.550274 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17261709-0872-4874-9ae5-afefade6a5f1 None None] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:42.550274 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17261709-0872-4874-9ae5-afefade6a5f1 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:42.550274 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17261709-0872-4874-9ae5-afefade6a5f1 None None] PATH_INFO: `/` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:42.551766 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 377/753] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 12:11:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:11:42.552658 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17261709-0872-4874-9ae5-afefade6a5f1 None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:42.556027 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f39faa-8298-4351-bda9-00a749e3cad2 None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:42.589332 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:11:42.596194 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78338043-4d56-4352-be91-a6d7ea7b3ad6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:42.597569 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78338043-4d56-4352-be91-a6d7ea7b3ad6 None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:42.597982 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78338043-4d56-4352-be91-a6d7ea7b3ad6 None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:42.598332 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78338043-4d56-4352-be91-a6d7ea7b3ad6 None admin] PATH_INFO: `/v3/domains` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:42.599041 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78338043-4d56-4352-be91-a6d7ea7b3ad6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81449) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:11:42.599407 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78338043-4d56-4352-be91-a6d7ea7b3ad6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:11:42.600974 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78338043-4d56-4352-be91-a6d7ea7b3ad6 None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:11:42.616893 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 377/754] 10.222.0.15 () {56 vars in 1109 bytes} [Thu Sep 15 12:11:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:11:42.616893 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4524ee-bdeb-45f8-8839-e84e324e5253 None None] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:42.616893 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4524ee-bdeb-45f8-8839-e84e324e5253 None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:42.616893 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4524ee-bdeb-45f8-8839-e84e324e5253 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:42.635035 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.auth.core [None req-fc4524ee-bdeb-45f8-8839-e84e324e5253 None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81449) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:11:42.636430 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-fc4524ee-bdeb-45f8-8839-e84e324e5253 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:11:42.668330 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17261709-0872-4874-9ae5-afefade6a5f1 None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:42.705233 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10eb989e-dbbf-4888-b9b9-299437677275 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:42.707471 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10eb989e-dbbf-4888-b9b9-299437677275 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:42.707471 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10eb989e-dbbf-4888-b9b9-299437677275 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:42.707471 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10eb989e-dbbf-4888-b9b9-299437677275 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:42.710071 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 378/755] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 12:11:42 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 15 12:11:42.711330 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4524ee-bdeb-45f8-8839-e84e324e5253 None None] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:42.722764 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10eb989e-dbbf-4888-b9b9-299437677275 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:11:42.724399 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10eb989e-dbbf-4888-b9b9-299437677275 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:11:42.740202 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:11:42.741667 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application [None req-10eb989e-dbbf-4888-b9b9-299437677275 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 12:11:42.741667 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:11:42.741667 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:11:42.741667 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:11:42.741667 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:11:42.741667 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:11:42.741667 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:11:42.741667 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:11:42.741667 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:11:42.741667 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:11:42.741667 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:11:42.741667 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:11:42.741667 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 12:11:42.741667 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 12:11:42.741667 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 12:11:42.741667 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 12:11:42.742594 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:11:42.742594 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:11:42.742594 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:11:42.742594 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:11:42.742594 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:11:42.742594 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:11:42.742594 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:11:42.742594 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application with Lock( Sep 15 12:11:42.742594 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:11:42.742594 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:11:42.742594 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:11:42.742594 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:11:42.742594 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:11:42.742594 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:11:42.742594 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:11:42.742594 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:11:42.742594 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 12:11:42.743486 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 12:11:42.743486 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 12:11:42.743486 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 15 12:11:42.745565 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 378/756] 10.222.0.15 () {56 vars in 1108 bytes} [Thu Sep 15 12:11:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:11:42.752649 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10eb989e-dbbf-4888-b9b9-299437677275 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:42.828340 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4524ee-bdeb-45f8-8839-e84e324e5253 None None] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:42.858298 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:11:42.867278 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52be6343-6f53-4ef1-a0ed-cb496cbf233c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:42.867847 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52be6343-6f53-4ef1-a0ed-cb496cbf233c None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:42.868121 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52be6343-6f53-4ef1-a0ed-cb496cbf233c None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:42.868402 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52be6343-6f53-4ef1-a0ed-cb496cbf233c None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:42.875303 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10eb989e-dbbf-4888-b9b9-299437677275 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:42.875603 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52be6343-6f53-4ef1-a0ed-cb496cbf233c None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:11:42.877143 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52be6343-6f53-4ef1-a0ed-cb496cbf233c None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:11:42.889501 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application [None req-52be6343-6f53-4ef1-a0ed-cb496cbf233c None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 15 12:11:42.889501 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:11:42.889501 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:11:42.889501 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:11:42.889501 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:11:42.889501 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:11:42.889501 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:11:42.889501 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:11:42.889501 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:11:42.889501 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:11:42.889501 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:11:42.889501 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:11:42.889501 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 15 12:11:42.889501 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 15 12:11:42.889501 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 15 12:11:42.889501 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 15 12:11:42.890556 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:11:42.890556 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:11:42.890556 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 15 12:11:42.890556 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 12:11:42.890556 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 15 12:11:42.890556 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 12:11:42.890556 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:11:42.890556 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:11:42.890556 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:11:42.890556 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:11:42.890556 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:11:42.890556 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application with Lock( Sep 15 12:11:42.890556 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:11:42.890556 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:11:42.890556 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:11:42.890556 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:11:42.890556 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:11:42.890556 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:11:42.892114 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:11:42.892114 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:11:42.892114 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 15 12:11:42.892114 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 15 12:11:42.892114 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 15 12:11:42.892114 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 15 12:11:42.892114 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 15 12:11:42.892114 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 15 12:11:42.892114 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 15 12:11:42.892114 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application Sep 15 12:11:42.892114 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 379/757] 10.222.0.15 () {56 vars in 1117 bytes} [Thu Sep 15 12:11:42 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:11:42.896921 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52be6343-6f53-4ef1-a0ed-cb496cbf233c None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:42.918766 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46cf4a94-6cb3-4744-9ea3-a2c13e90dee5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:42.919789 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cf4a94-6cb3-4744-9ea3-a2c13e90dee5 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:42.920274 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cf4a94-6cb3-4744-9ea3-a2c13e90dee5 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:42.920799 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cf4a94-6cb3-4744-9ea3-a2c13e90dee5 None admin] PATH_INFO: `/v3/domains` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:42.921792 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46cf4a94-6cb3-4744-9ea3-a2c13e90dee5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:11:42.922530 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46cf4a94-6cb3-4744-9ea3-a2c13e90dee5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:11:42.924808 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46cf4a94-6cb3-4744-9ea3-a2c13e90dee5 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:11:42.952422 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 379/758] 10.222.0.15 () {56 vars in 1109 bytes} [Thu Sep 15 12:11:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 12:11:42.961399 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46cf4a94-6cb3-4744-9ea3-a2c13e90dee5 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:43.029123 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:11:43.037537 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52be6343-6f53-4ef1-a0ed-cb496cbf233c None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:43.081141 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9088a7b-fb69-45cd-88b8-a0a9dc26b734 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:43.081860 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9088a7b-fb69-45cd-88b8-a0a9dc26b734 None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:43.082139 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9088a7b-fb69-45cd-88b8-a0a9dc26b734 None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:43.082446 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9088a7b-fb69-45cd-88b8-a0a9dc26b734 None admin] PATH_INFO: `/v3/users` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:43.083224 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9088a7b-fb69-45cd-88b8-a0a9dc26b734 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81449) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:11:43.083580 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9088a7b-fb69-45cd-88b8-a0a9dc26b734 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:11:43.085301 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9088a7b-fb69-45cd-88b8-a0a9dc26b734 None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:11:43.106225 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 380/759] 10.222.0.15 () {56 vars in 1149 bytes} [Thu Sep 15 12:11:42 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 12:11:43.107613 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9088a7b-fb69-45cd-88b8-a0a9dc26b734 None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:43.138706 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46cf4a94-6cb3-4744-9ea3-a2c13e90dee5 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:43.184278 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59cff357-1585-4e87-a56f-3744fbd7842c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:43.185139 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cff357-1585-4e87-a56f-3744fbd7842c None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:43.185515 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cff357-1585-4e87-a56f-3744fbd7842c None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:43.185792 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cff357-1585-4e87-a56f-3744fbd7842c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:43.193738 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59cff357-1585-4e87-a56f-3744fbd7842c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:11:43.195287 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59cff357-1585-4e87-a56f-3744fbd7842c None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:11:43.209347 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application [None req-59cff357-1585-4e87-a56f-3744fbd7842c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 12:11:43.209347 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:11:43.209347 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:11:43.209347 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:11:43.209347 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:11:43.209347 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:11:43.209347 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:11:43.209347 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:11:43.209347 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:11:43.209347 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:11:43.209347 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:11:43.209347 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:11:43.209347 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 15 12:11:43.209347 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 15 12:11:43.209347 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 15 12:11:43.210280 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 15 12:11:43.210280 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:11:43.210280 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:11:43.210280 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:11:43.210280 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:11:43.210280 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:11:43.210280 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:11:43.210280 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:11:43.210280 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application with Lock( Sep 15 12:11:43.210280 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:11:43.210280 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:11:43.210280 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:11:43.210280 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:11:43.210280 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:11:43.210280 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:11:43.210280 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:11:43.210280 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:11:43.211777 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 15 12:11:43.211777 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 15 12:11:43.211777 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 15 12:11:43.211777 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 15 12:11:43.211777 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 15 12:11:43.211777 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 15 12:11:43.211777 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 12:11:43.211777 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 15 12:11:43.211777 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 380/760] 10.222.0.15 () {56 vars in 1111 bytes} [Thu Sep 15 12:11:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 15 12:11:43.211777 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59cff357-1585-4e87-a56f-3744fbd7842c None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:43.216641 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9088a7b-fb69-45cd-88b8-a0a9dc26b734 None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:43.221723 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:11:43.256421 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e77ebb-6ffd-4cd3-9f3c-38df514f7d1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:43.257144 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e77ebb-6ffd-4cd3-9f3c-38df514f7d1f None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:43.257422 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e77ebb-6ffd-4cd3-9f3c-38df514f7d1f None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:43.257690 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e77ebb-6ffd-4cd3-9f3c-38df514f7d1f None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:43.258274 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e77ebb-6ffd-4cd3-9f3c-38df514f7d1f None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:11:43.259802 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e77ebb-6ffd-4cd3-9f3c-38df514f7d1f None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:11:43.269656 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application [None req-62e77ebb-6ffd-4cd3-9f3c-38df514f7d1f None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 15 12:11:43.269656 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:11:43.269656 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 15 12:11:43.269656 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 15 12:11:43.269656 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 15 12:11:43.269656 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 15 12:11:43.269656 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 15 12:11:43.269656 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 15 12:11:43.269656 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 15 12:11:43.269656 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application Sep 15 12:11:43.269656 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 15 12:11:43.269656 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application Sep 15 12:11:43.269656 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:11:43.269656 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:11:43.269656 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:11:43.269656 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:11:43.269656 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:11:43.272057 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:11:43.272057 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:11:43.272057 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:11:43.272057 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:11:43.272057 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:11:43.272057 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:11:43.272057 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 15 12:11:43.272057 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 15 12:11:43.272057 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 15 12:11:43.272057 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 15 12:11:43.272057 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:11:43.272057 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:11:43.272057 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:11:43.272057 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:11:43.272057 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:11:43.272057 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:11:43.274070 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:11:43.274070 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application with Lock( Sep 15 12:11:43.274070 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:11:43.274070 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:11:43.274070 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:11:43.274070 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:11:43.274070 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:11:43.274070 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:11:43.274070 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:11:43.274070 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:11:43.274070 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 15 12:11:43.274070 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 15 12:11:43.274070 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 15 12:11:43.274070 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application Sep 15 12:11:43.281432 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 381/761] 10.222.0.15 () {56 vars in 1129 bytes} [Thu Sep 15 12:11:43 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:11:43.281432 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e77ebb-6ffd-4cd3-9f3c-38df514f7d1f None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:43.291271 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:11:43.330985 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59cff357-1585-4e87-a56f-3744fbd7842c None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:43.371225 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44bdce9c-ab8c-41dc-bd58-cca567d9b560 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:43.371900 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bdce9c-ab8c-41dc-bd58-cca567d9b560 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:43.372177 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bdce9c-ab8c-41dc-bd58-cca567d9b560 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:43.372446 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bdce9c-ab8c-41dc-bd58-cca567d9b560 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:43.395877 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e77ebb-6ffd-4cd3-9f3c-38df514f7d1f None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:43.433020 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c74bc5d-1d22-4b47-b0f2-572113fbe4ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:43.434016 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c74bc5d-1d22-4b47-b0f2-572113fbe4ed None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:43.434629 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c74bc5d-1d22-4b47-b0f2-572113fbe4ed None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:43.435120 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c74bc5d-1d22-4b47-b0f2-572113fbe4ed None admin] PATH_INFO: `/v3/projects` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:43.436059 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c74bc5d-1d22-4b47-b0f2-572113fbe4ed None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81449) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:11:43.436558 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c74bc5d-1d22-4b47-b0f2-572113fbe4ed None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:11:43.438445 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c74bc5d-1d22-4b47-b0f2-572113fbe4ed None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:11:43.448157 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44bdce9c-ab8c-41dc-bd58-cca567d9b560 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:11:43.449700 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44bdce9c-ab8c-41dc-bd58-cca567d9b560 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:11:43.451938 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 382/762] 10.222.0.15 () {56 vars in 1148 bytes} [Thu Sep 15 12:11:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:11:43.453482 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c74bc5d-1d22-4b47-b0f2-572113fbe4ed None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:43.458741 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:11:43.529950 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:11:43.571385 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c74bc5d-1d22-4b47-b0f2-572113fbe4ed None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:43.576475 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 381/763] 10.222.0.15 () {58 vars in 1282 bytes} [Thu Sep 15 12:11:43 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 15 12:11:43.578380 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44bdce9c-ab8c-41dc-bd58-cca567d9b560 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:43.626965 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9900224-f17e-40a2-b0c8-fe1e7f932dc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:43.627583 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9900224-f17e-40a2-b0c8-fe1e7f932dc1 None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:43.627862 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9900224-f17e-40a2-b0c8-fe1e7f932dc1 None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:43.628133 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9900224-f17e-40a2-b0c8-fe1e7f932dc1 None admin] PATH_INFO: `/v3/services` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:43.628809 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9900224-f17e-40a2-b0c8-fe1e7f932dc1 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81449) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:11:43.629089 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9900224-f17e-40a2-b0c8-fe1e7f932dc1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:11:43.631031 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9900224-f17e-40a2-b0c8-fe1e7f932dc1 None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:11:43.637444 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 383/764] 10.222.0.15 () {56 vars in 1124 bytes} [Thu Sep 15 12:11:43 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:11:43.638296 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9900224-f17e-40a2-b0c8-fe1e7f932dc1 None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:43.718171 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44bdce9c-ab8c-41dc-bd58-cca567d9b560 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:43.729732 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:11:43.763047 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9900224-f17e-40a2-b0c8-fe1e7f932dc1 None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:43.765277 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8241523b-47f0-4857-8a88-c218fe9b3268 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:43.766167 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8241523b-47f0-4857-8a88-c218fe9b3268 None admin] REQUEST_METHOD: `PUT` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:43.766668 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8241523b-47f0-4857-8a88-c218fe9b3268 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:43.767165 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8241523b-47f0-4857-8a88-c218fe9b3268 None admin] PATH_INFO: `/v3/projects/540a32ef88764e8c8d9201ae22f6ddaf/users/eface89e261840a2abb4206be26455c7/roles/9c769b5c82884766b2c510b1f0f1364c` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:43.769282 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8241523b-47f0-4857-8a88-c218fe9b3268 None admin] RBAC: Authorizing `identity:create_grant(project_id=540a32ef88764e8c8d9201ae22f6ddaf, user_id=eface89e261840a2abb4206be26455c7, role_id=9c769b5c82884766b2c510b1f0f1364c)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:11:43.771500 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8241523b-47f0-4857-8a88-c218fe9b3268 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:11:43.790987 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 382/765] 10.222.0.15 () {58 vars in 1439 bytes} [Thu Sep 15 12:11:43 2022] PUT /identity/v3/projects/540a32ef88764e8c8d9201ae22f6ddaf/users/eface89e261840a2abb4206be26455c7/roles/9c769b5c82884766b2c510b1f0f1364c => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:11:43.792139 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8241523b-47f0-4857-8a88-c218fe9b3268 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:43.812818 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738d8c70-8e23-462e-aacf-c64f9b4be907 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:43.814018 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738d8c70-8e23-462e-aacf-c64f9b4be907 None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:43.814307 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738d8c70-8e23-462e-aacf-c64f9b4be907 None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:43.814578 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738d8c70-8e23-462e-aacf-c64f9b4be907 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:43.817605 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:11:43.830151 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-738d8c70-8e23-462e-aacf-c64f9b4be907 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:11:43.831611 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-738d8c70-8e23-462e-aacf-c64f9b4be907 None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:11:43.848915 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application [None req-738d8c70-8e23-462e-aacf-c64f9b4be907 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 12:11:43.848915 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:11:43.848915 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:11:43.848915 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:11:43.848915 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:11:43.848915 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:11:43.848915 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:11:43.848915 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:11:43.848915 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:11:43.848915 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:11:43.848915 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:11:43.848915 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:11:43.848915 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 12:11:43.848915 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 12:11:43.848915 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 12:11:43.848915 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 12:11:43.850006 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:11:43.850006 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:11:43.850006 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:11:43.850006 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:11:43.850006 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:11:43.850006 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:11:43.850006 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:11:43.850006 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application with Lock( Sep 15 12:11:43.850006 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:11:43.850006 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:11:43.850006 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:11:43.850006 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:11:43.850006 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:11:43.850006 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:11:43.850006 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:11:43.850006 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:11:43.850006 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 12:11:43.851052 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 12:11:43.851052 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 12:11:43.851052 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application Sep 15 12:11:43.851052 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 384/766] 10.222.0.15 () {56 vars in 1108 bytes} [Thu Sep 15 12:11:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:11:43.852018 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0496e3e1-5edd-4ab9-979c-4259b1c195ca None None] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:43.852018 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0496e3e1-5edd-4ab9-979c-4259b1c195ca None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:43.852018 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0496e3e1-5edd-4ab9-979c-4259b1c195ca None None] PATH_INFO: `/` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:43.853274 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 385/767] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 12:11:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:11:43.854789 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8842b622-3205-43e1-999e-90380f811c5e None None] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:43.854983 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8842b622-3205-43e1-999e-90380f811c5e None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:43.855214 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8842b622-3205-43e1-999e-90380f811c5e None None] PATH_INFO: `/` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:43.856788 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 386/768] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 12:11:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:11:43.857307 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8842b622-3205-43e1-999e-90380f811c5e None None] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:43.917787 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8241523b-47f0-4857-8a88-c218fe9b3268 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:43.956880 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be378a5-9bf9-4be2-845e-3481ba363bcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:43.958030 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be378a5-9bf9-4be2-845e-3481ba363bcf None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:43.958311 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be378a5-9bf9-4be2-845e-3481ba363bcf None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:43.958643 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be378a5-9bf9-4be2-845e-3481ba363bcf None admin] PATH_INFO: `/v3/services` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:43.959280 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be378a5-9bf9-4be2-845e-3481ba363bcf None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:11:43.959607 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be378a5-9bf9-4be2-845e-3481ba363bcf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:11:43.961026 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be378a5-9bf9-4be2-845e-3481ba363bcf None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:11:43.970673 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 383/769] 10.222.0.15 () {56 vars in 1124 bytes} [Thu Sep 15 12:11:43 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:11:43.972199 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18699e0-ab71-4a81-aef2-30d5863b14f5 None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:43.972487 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18699e0-ab71-4a81-aef2-30d5863b14f5 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:43.972840 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18699e0-ab71-4a81-aef2-30d5863b14f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:43.989007 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8842b622-3205-43e1-999e-90380f811c5e None None] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:43.993971 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-c18699e0-ab71-4a81-aef2-30d5863b14f5 None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:11:43.995165 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-c18699e0-ab71-4a81-aef2-30d5863b14f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:11:44.018908 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:11:44.028437 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bce2f90-8cad-4fea-b3e9-e3fd9e8ded7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:44.029081 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bce2f90-8cad-4fea-b3e9-e3fd9e8ded7f None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:44.029362 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bce2f90-8cad-4fea-b3e9-e3fd9e8ded7f None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:44.029634 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bce2f90-8cad-4fea-b3e9-e3fd9e8ded7f None admin] PATH_INFO: `/v3/domains` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:44.030267 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bce2f90-8cad-4fea-b3e9-e3fd9e8ded7f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81449) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:11:44.030569 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bce2f90-8cad-4fea-b3e9-e3fd9e8ded7f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:11:44.031989 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bce2f90-8cad-4fea-b3e9-e3fd9e8ded7f None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:11:44.048640 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 387/770] 10.222.0.15 () {56 vars in 1109 bytes} [Thu Sep 15 12:11:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:11:44.048640 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078595c3-5b88-481f-9f15-ccfd43df2e46 None None] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:44.048640 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078595c3-5b88-481f-9f15-ccfd43df2e46 None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:44.048640 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078595c3-5b88-481f-9f15-ccfd43df2e46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:44.066110 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.auth.core [None req-078595c3-5b88-481f-9f15-ccfd43df2e46 None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81449) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:11:44.067874 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-078595c3-5b88-481f-9f15-ccfd43df2e46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:11:44.116990 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:11:44.152796 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 388/771] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 12:11:44 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 15 12:11:44.152796 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078595c3-5b88-481f-9f15-ccfd43df2e46 None None] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:44.178972 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 384/772] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 12:11:43 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 15 12:11:44.179661 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18699e0-ab71-4a81-aef2-30d5863b14f5 None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:44.307096 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078595c3-5b88-481f-9f15-ccfd43df2e46 None None] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:44.361265 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18699e0-ab71-4a81-aef2-30d5863b14f5 None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:44.372703 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97873c82-9c73-4541-8bcf-5315f2680e3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:44.373349 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97873c82-9c73-4541-8bcf-5315f2680e3a None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:44.373627 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97873c82-9c73-4541-8bcf-5315f2680e3a None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:44.373900 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97873c82-9c73-4541-8bcf-5315f2680e3a None admin] PATH_INFO: `/v3/endpoints` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:44.374602 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97873c82-9c73-4541-8bcf-5315f2680e3a None admin] RBAC: Adding query filter params (interface=public, service_id=73a44de282f64bd8a8dc23e6d128037a, region_id=RegionOne) {{(pid=81449) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:11:44.374906 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97873c82-9c73-4541-8bcf-5315f2680e3a None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:11:44.376294 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97873c82-9c73-4541-8bcf-5315f2680e3a None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:11:44.391341 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 389/773] 10.222.0.15 () {56 vars in 1251 bytes} [Thu Sep 15 12:11:44 2022] GET /identity/v3/endpoints?service_id=73a44de282f64bd8a8dc23e6d128037a&interface=public®ion_id=RegionOne => generated 194 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:11:44.396056 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97873c82-9c73-4541-8bcf-5315f2680e3a None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:44.428442 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487e0ff3-ea14-451b-bf17-07409c8030f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:44.429337 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487e0ff3-ea14-451b-bf17-07409c8030f0 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:44.429622 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487e0ff3-ea14-451b-bf17-07409c8030f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:44.430717 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487e0ff3-ea14-451b-bf17-07409c8030f0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:44.442523 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487e0ff3-ea14-451b-bf17-07409c8030f0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:11:44.445000 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487e0ff3-ea14-451b-bf17-07409c8030f0 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:11:44.467275 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application [None req-487e0ff3-ea14-451b-bf17-07409c8030f0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 12:11:44.467275 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:11:44.467275 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:11:44.467275 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:11:44.467275 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:11:44.467275 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:11:44.467275 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:11:44.467275 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:11:44.467275 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:11:44.467275 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:11:44.467275 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:11:44.467275 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:11:44.467275 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 15 12:11:44.467275 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 15 12:11:44.467275 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 15 12:11:44.468225 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 15 12:11:44.468225 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:11:44.468225 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:11:44.468225 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:11:44.468225 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:11:44.468225 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:11:44.468225 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:11:44.468225 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:11:44.468225 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application with Lock( Sep 15 12:11:44.468225 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:11:44.468225 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:11:44.468225 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:11:44.468225 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:11:44.468225 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:11:44.468225 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:11:44.468225 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:11:44.468225 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:11:44.469197 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 15 12:11:44.469197 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 15 12:11:44.469197 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 15 12:11:44.469197 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 15 12:11:44.469197 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 15 12:11:44.469197 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 15 12:11:44.469197 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 12:11:44.469197 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 15 12:11:44.469197 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 385/774] 10.222.0.15 () {56 vars in 1111 bytes} [Thu Sep 15 12:11:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 289 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:11:44.469197 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487e0ff3-ea14-451b-bf17-07409c8030f0 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:44.479053 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:11:44.496674 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:11:44.541756 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97873c82-9c73-4541-8bcf-5315f2680e3a None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:44.574878 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18265eb5-10d1-4c37-aab1-595310c78706 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:44.575409 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18265eb5-10d1-4c37-aab1-595310c78706 None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:44.575690 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18265eb5-10d1-4c37-aab1-595310c78706 None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:44.575923 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487e0ff3-ea14-451b-bf17-07409c8030f0 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:44.576028 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18265eb5-10d1-4c37-aab1-595310c78706 None admin] PATH_INFO: `/v3/services` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:44.576560 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18265eb5-10d1-4c37-aab1-595310c78706 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:11:44.579056 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18265eb5-10d1-4c37-aab1-595310c78706 None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:11:44.592845 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 390/775] 10.222.0.15 () {56 vars in 1087 bytes} [Thu Sep 15 12:11:44 2022] GET /identity/v3/services => generated 772 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:11:44.592845 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18265eb5-10d1-4c37-aab1-595310c78706 None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:44.625419 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a581252-1d92-47f8-b9cc-fd1f5b124d99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:44.626139 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a581252-1d92-47f8-b9cc-fd1f5b124d99 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:44.626422 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a581252-1d92-47f8-b9cc-fd1f5b124d99 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:44.626693 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a581252-1d92-47f8-b9cc-fd1f5b124d99 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:44.640642 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a581252-1d92-47f8-b9cc-fd1f5b124d99 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 12:11:44.646255 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application [None req-9a581252-1d92-47f8-b9cc-fd1f5b124d99 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 15 12:11:44.646255 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:11:44.646255 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:11:44.646255 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:11:44.646255 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:11:44.646255 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:11:44.646255 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:11:44.646255 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:11:44.646255 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:11:44.646255 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:11:44.646255 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:11:44.646255 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:11:44.646255 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 12:11:44.646255 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 12:11:44.646255 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 12:11:44.646255 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 12:11:44.647218 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 12:11:44.647218 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 12:11:44.647218 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 12:11:44.647218 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 12:11:44.647218 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:11:44.647218 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:11:44.647218 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:11:44.647218 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:11:44.647218 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:11:44.647218 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:11:44.647218 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:11:44.647218 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application with Lock( Sep 15 12:11:44.647218 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:11:44.647218 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:11:44.647218 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:11:44.647218 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:11:44.648062 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:11:44.648062 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:11:44.648062 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:11:44.648062 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:11:44.648062 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 12:11:44.648062 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 12:11:44.648062 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 12:11:44.648062 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 12:11:44.648062 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 12:11:44.648062 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 12:11:44.648062 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 15 12:11:44.648062 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 15 12:11:44.648062 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 386/776] 10.222.0.15 () {56 vars in 1096 bytes} [Thu Sep 15 12:11:44 2022] GET /identity/v3/roles/admin => generated 83 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:11:44.649375 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a581252-1d92-47f8-b9cc-fd1f5b124d99 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:44.730742 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18265eb5-10d1-4c37-aab1-595310c78706 None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:44.733477 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:11:44.765423 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:11:44.772355 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93186d3f-0b91-4bc5-8861-48514beb4c38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:44.773364 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93186d3f-0b91-4bc5-8861-48514beb4c38 None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:44.773833 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93186d3f-0b91-4bc5-8861-48514beb4c38 None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:44.774355 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93186d3f-0b91-4bc5-8861-48514beb4c38 None admin] PATH_INFO: `/v3/projects` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:44.775233 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93186d3f-0b91-4bc5-8861-48514beb4c38 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81449) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:11:44.775738 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93186d3f-0b91-4bc5-8861-48514beb4c38 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:11:44.777414 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93186d3f-0b91-4bc5-8861-48514beb4c38 None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:11:44.790118 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a581252-1d92-47f8-b9cc-fd1f5b124d99 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:44.794331 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 391/777] 10.222.0.15 () {56 vars in 1148 bytes} [Thu Sep 15 12:11:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:11:44.795451 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93186d3f-0b91-4bc5-8861-48514beb4c38 None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:44.834305 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf07c35-320f-458a-9f47-5a6cf6a071a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:44.834862 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf07c35-320f-458a-9f47-5a6cf6a071a3 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:44.835140 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf07c35-320f-458a-9f47-5a6cf6a071a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:44.835412 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf07c35-320f-458a-9f47-5a6cf6a071a3 None admin] PATH_INFO: `/v3/services/network` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:44.835994 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf07c35-320f-458a-9f47-5a6cf6a071a3 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:11:44.838112 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf07c35-320f-458a-9f47-5a6cf6a071a3 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:11:44.851314 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application [None req-1bf07c35-320f-458a-9f47-5a6cf6a071a3 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 15 12:11:44.851314 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:11:44.851314 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 15 12:11:44.851314 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 15 12:11:44.851314 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 15 12:11:44.851314 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 15 12:11:44.851314 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 15 12:11:44.851314 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 15 12:11:44.851314 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 15 12:11:44.851314 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 15 12:11:44.851314 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 15 12:11:44.851314 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 15 12:11:44.851314 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:11:44.851314 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:11:44.851314 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:11:44.851314 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:11:44.851314 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:11:44.852343 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:11:44.852343 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:11:44.852343 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:11:44.852343 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:11:44.852343 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:11:44.852343 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:11:44.852343 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 15 12:11:44.852343 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 15 12:11:44.852343 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 15 12:11:44.852343 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 15 12:11:44.852343 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:11:44.852343 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:11:44.852343 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:11:44.852343 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:11:44.852343 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:11:44.852343 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:11:44.853275 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:11:44.853275 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application with Lock( Sep 15 12:11:44.853275 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:11:44.853275 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:11:44.853275 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:11:44.853275 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:11:44.853275 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:11:44.853275 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:11:44.853275 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:11:44.853275 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:11:44.853275 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 15 12:11:44.853275 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 15 12:11:44.853275 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 15 12:11:44.853275 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 15 12:11:44.859557 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 387/778] 10.222.0.15 () {56 vars in 1111 bytes} [Thu Sep 15 12:11:44 2022] GET /identity/v3/services/network => generated 88 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:11:44.859557 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf07c35-320f-458a-9f47-5a6cf6a071a3 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:44.908337 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93186d3f-0b91-4bc5-8861-48514beb4c38 None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:44.943781 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2c8213-c622-4432-bae2-e7b52f17ddcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:44.944394 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2c8213-c622-4432-bae2-e7b52f17ddcf None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:44.944672 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2c8213-c622-4432-bae2-e7b52f17ddcf None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:44.944988 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2c8213-c622-4432-bae2-e7b52f17ddcf None admin] PATH_INFO: `/v3/roles` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:44.945701 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2c8213-c622-4432-bae2-e7b52f17ddcf None admin] RBAC: Adding query filter params (name=admin) {{(pid=81449) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:11:44.946002 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2c8213-c622-4432-bae2-e7b52f17ddcf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:11:44.947411 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2c8213-c622-4432-bae2-e7b52f17ddcf None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:11:44.959143 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 392/779] 10.222.0.15 () {56 vars in 1099 bytes} [Thu Sep 15 12:11:44 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:11:44.960061 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2c8213-c622-4432-bae2-e7b52f17ddcf None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:44.995329 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:11:44.999964 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf07c35-320f-458a-9f47-5a6cf6a071a3 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:45.036302 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee333f7-938c-44dc-a51a-1fc83e748c8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:45.037087 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee333f7-938c-44dc-a51a-1fc83e748c8a None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:45.037458 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee333f7-938c-44dc-a51a-1fc83e748c8a None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:45.037799 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee333f7-938c-44dc-a51a-1fc83e748c8a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:45.038524 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee333f7-938c-44dc-a51a-1fc83e748c8a None admin] RBAC: Adding query filter params (role.id=e1cecc29e028497f9cdaa04268c6a6ec, scope.project.id=540a32ef88764e8c8d9201ae22f6ddaf, user.id=527f4dd6031546c785bc6360dacb60af) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:11:45.038830 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee333f7-938c-44dc-a51a-1fc83e748c8a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:11:45.040255 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee333f7-938c-44dc-a51a-1fc83e748c8a None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:11:45.047124 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 388/780] 10.222.0.15 () {56 vars in 1374 bytes} [Thu Sep 15 12:11:44 2022] GET /identity/v3/role_assignments?user.id=527f4dd6031546c785bc6360dacb60af&scope.project.id=540a32ef88764e8c8d9201ae22f6ddaf&role.id=e1cecc29e028497f9cdaa04268c6a6ec => generated 259 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:11:45.048318 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee333f7-938c-44dc-a51a-1fc83e748c8a None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:45.064657 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:11:45.112366 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2c8213-c622-4432-bae2-e7b52f17ddcf None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:45.172288 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f25e77d-d388-4c2f-a1f1-682b029f5954 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:45.172941 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f25e77d-d388-4c2f-a1f1-682b029f5954 None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:45.173226 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f25e77d-d388-4c2f-a1f1-682b029f5954 None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:45.173501 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f25e77d-d388-4c2f-a1f1-682b029f5954 None admin] PATH_INFO: `/v3/services` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:45.174149 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f25e77d-d388-4c2f-a1f1-682b029f5954 None admin] RBAC: Adding query filter params (name=network) {{(pid=81449) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:11:45.174454 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f25e77d-d388-4c2f-a1f1-682b029f5954 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:11:45.175849 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f25e77d-d388-4c2f-a1f1-682b029f5954 None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:11:45.181490 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 393/781] 10.222.0.15 () {56 vars in 1112 bytes} [Thu Sep 15 12:11:44 2022] GET /identity/v3/services?name=network => generated 124 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:11:45.187114 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f25e77d-d388-4c2f-a1f1-682b029f5954 None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:45.226610 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee333f7-938c-44dc-a51a-1fc83e748c8a None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:45.290038 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b886702-03da-4343-8a5e-11f9855bdc29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:45.290685 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b886702-03da-4343-8a5e-11f9855bdc29 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:45.292152 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b886702-03da-4343-8a5e-11f9855bdc29 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:45.292438 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b886702-03da-4343-8a5e-11f9855bdc29 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:45.314719 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b886702-03da-4343-8a5e-11f9855bdc29 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:11:45.316158 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b886702-03da-4343-8a5e-11f9855bdc29 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:11:45.346297 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application [None req-2b886702-03da-4343-8a5e-11f9855bdc29 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 12:11:45.346297 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:11:45.346297 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:11:45.346297 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:11:45.346297 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:11:45.346297 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:11:45.346297 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:11:45.346297 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:11:45.346297 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:11:45.346297 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:11:45.346297 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:11:45.346297 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:11:45.346297 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 12:11:45.346297 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 12:11:45.346297 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 12:11:45.346297 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 12:11:45.347310 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:11:45.347310 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:11:45.347310 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:11:45.347310 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:11:45.347310 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:11:45.347310 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:11:45.347310 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:11:45.347310 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application with Lock( Sep 15 12:11:45.347310 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:11:45.347310 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:11:45.347310 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:11:45.347310 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:11:45.347310 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:11:45.347310 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:11:45.347310 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:11:45.347310 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:11:45.347310 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 12:11:45.348827 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 12:11:45.348827 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 12:11:45.348827 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 15 12:11:45.348827 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 389/782] 10.222.0.15 () {56 vars in 1108 bytes} [Thu Sep 15 12:11:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 300 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:11:45.353661 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b886702-03da-4343-8a5e-11f9855bdc29 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:45.376597 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:11:45.392138 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f25e77d-d388-4c2f-a1f1-682b029f5954 None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:45.452597 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecdb7166-1c25-46de-9034-5ec141db540c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:45.453236 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdb7166-1c25-46de-9034-5ec141db540c None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:45.453512 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdb7166-1c25-46de-9034-5ec141db540c None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:45.453776 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdb7166-1c25-46de-9034-5ec141db540c None admin] PATH_INFO: `/v3/services` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:45.454400 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecdb7166-1c25-46de-9034-5ec141db540c None admin] RBAC: Adding query filter params (type=network) {{(pid=81449) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:11:45.454706 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecdb7166-1c25-46de-9034-5ec141db540c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:11:45.456053 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecdb7166-1c25-46de-9034-5ec141db540c None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:11:45.470259 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 394/783] 10.222.0.15 () {56 vars in 1112 bytes} [Thu Sep 15 12:11:45 2022] GET /identity/v3/services?type=network => generated 350 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:11:45.471494 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecdb7166-1c25-46de-9034-5ec141db540c None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:45.481641 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:11:45.546898 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b886702-03da-4343-8a5e-11f9855bdc29 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:45.607690 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8ff508-0568-41a7-9d28-ed87328e5dc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:45.608308 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8ff508-0568-41a7-9d28-ed87328e5dc3 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:45.608581 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8ff508-0568-41a7-9d28-ed87328e5dc3 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:45.608872 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8ff508-0568-41a7-9d28-ed87328e5dc3 None admin] PATH_INFO: `/v3/domains` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:45.609513 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8ff508-0568-41a7-9d28-ed87328e5dc3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:11:45.609815 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8ff508-0568-41a7-9d28-ed87328e5dc3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:11:45.611165 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8ff508-0568-41a7-9d28-ed87328e5dc3 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:11:45.630332 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 390/784] 10.222.0.15 () {56 vars in 1109 bytes} [Thu Sep 15 12:11:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:11:45.639250 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8ff508-0568-41a7-9d28-ed87328e5dc3 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:45.663198 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecdb7166-1c25-46de-9034-5ec141db540c None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:45.726597 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dfa0376-f3bd-40c5-b0ba-3ba1e2acb763 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:45.727479 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfa0376-f3bd-40c5-b0ba-3ba1e2acb763 None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:45.727762 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfa0376-f3bd-40c5-b0ba-3ba1e2acb763 None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:45.728030 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfa0376-f3bd-40c5-b0ba-3ba1e2acb763 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:45.728729 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dfa0376-f3bd-40c5-b0ba-3ba1e2acb763 None admin] RBAC: Adding query filter params (interface=public, service_id=d6a7565cda9d4d17868233337730e0fb, region_id=RegionOne) {{(pid=81449) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:11:45.729059 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dfa0376-f3bd-40c5-b0ba-3ba1e2acb763 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:11:45.730712 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dfa0376-f3bd-40c5-b0ba-3ba1e2acb763 None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:11:45.740831 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 395/785] 10.222.0.15 () {56 vars in 1251 bytes} [Thu Sep 15 12:11:45 2022] GET /identity/v3/endpoints?service_id=d6a7565cda9d4d17868233337730e0fb&interface=public®ion_id=RegionOne => generated 194 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:11:45.745834 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dfa0376-f3bd-40c5-b0ba-3ba1e2acb763 None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:45.788746 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:11:45.826714 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8ff508-0568-41a7-9d28-ed87328e5dc3 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:45.848049 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:11:45.868781 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd0bb7a-ea66-40bf-8ca8-f1e5b2b56b8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:45.868781 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd0bb7a-ea66-40bf-8ca8-f1e5b2b56b8c None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:45.868781 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd0bb7a-ea66-40bf-8ca8-f1e5b2b56b8c None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:45.868781 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd0bb7a-ea66-40bf-8ca8-f1e5b2b56b8c None admin] PATH_INFO: `/v3/users/swift` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:45.872809 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd0bb7a-ea66-40bf-8ca8-f1e5b2b56b8c None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:11:45.874341 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd0bb7a-ea66-40bf-8ca8-f1e5b2b56b8c None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:11:45.887013 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application [None req-8dd0bb7a-ea66-40bf-8ca8-f1e5b2b56b8c None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 15 12:11:45.887013 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:11:45.887013 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:11:45.887013 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:11:45.887013 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:11:45.887013 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:11:45.887013 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:11:45.887013 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:11:45.887013 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:11:45.887013 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:11:45.887013 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:11:45.887013 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:11:45.887013 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 15 12:11:45.887013 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 15 12:11:45.887013 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 15 12:11:45.887013 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 15 12:11:45.888177 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:11:45.888177 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:11:45.888177 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 15 12:11:45.888177 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 12:11:45.888177 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 15 12:11:45.888177 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 12:11:45.888177 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:11:45.888177 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:11:45.888177 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:11:45.888177 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:11:45.888177 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:11:45.888177 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application with Lock( Sep 15 12:11:45.888177 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:11:45.888177 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:11:45.888177 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:11:45.888177 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:11:45.888177 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:11:45.888177 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:11:45.889425 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:11:45.889425 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:11:45.889425 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 15 12:11:45.889425 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 15 12:11:45.889425 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 15 12:11:45.889425 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 15 12:11:45.889425 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 15 12:11:45.889425 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 15 12:11:45.889425 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 15 12:11:45.889425 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 15 12:11:45.891199 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 391/786] 10.222.0.15 () {56 vars in 1096 bytes} [Thu Sep 15 12:11:45 2022] GET /identity/v3/users/swift => generated 83 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:11:45.897776 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd0bb7a-ea66-40bf-8ca8-f1e5b2b56b8c None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:45.927346 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dfa0376-f3bd-40c5-b0ba-3ba1e2acb763 None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:45.972248 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5472e1-17af-45fc-a6e7-09b3c611aa75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:45.972910 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5472e1-17af-45fc-a6e7-09b3c611aa75 None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:45.973209 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5472e1-17af-45fc-a6e7-09b3c611aa75 None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:45.973467 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5472e1-17af-45fc-a6e7-09b3c611aa75 None admin] PATH_INFO: `/v3/services` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:45.974081 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5472e1-17af-45fc-a6e7-09b3c611aa75 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:11:45.975455 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5472e1-17af-45fc-a6e7-09b3c611aa75 None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:11:45.983529 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 396/787] 10.222.0.15 () {56 vars in 1087 bytes} [Thu Sep 15 12:11:45 2022] GET /identity/v3/services => generated 772 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:11:46.075286 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd0bb7a-ea66-40bf-8ca8-f1e5b2b56b8c None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:46.125908 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a390b7-5b85-4e47-8dfd-44d5c3481f00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:46.126549 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a390b7-5b85-4e47-8dfd-44d5c3481f00 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:46.126829 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a390b7-5b85-4e47-8dfd-44d5c3481f00 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:46.127103 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a390b7-5b85-4e47-8dfd-44d5c3481f00 None admin] PATH_INFO: `/v3/users` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:46.127843 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a390b7-5b85-4e47-8dfd-44d5c3481f00 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:11:46.128150 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a390b7-5b85-4e47-8dfd-44d5c3481f00 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:11:46.130224 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a390b7-5b85-4e47-8dfd-44d5c3481f00 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:11:46.160242 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 392/788] 10.222.0.15 () {56 vars in 1135 bytes} [Thu Sep 15 12:11:45 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:11:46.166143 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5472e1-17af-45fc-a6e7-09b3c611aa75 None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:46.210511 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c642605-6255-4378-980e-d37b2e4dc858 None None] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:46.210839 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c642605-6255-4378-980e-d37b2e4dc858 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:46.211081 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c642605-6255-4378-980e-d37b2e4dc858 None None] PATH_INFO: `/` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:46.212668 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 393/789] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 12:11:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 15 12:11:46.217486 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf80375-0d5b-4ebd-893f-cf0d92d8e54e None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:46.217880 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf80375-0d5b-4ebd-893f-cf0d92d8e54e None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:46.218275 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf80375-0d5b-4ebd-893f-cf0d92d8e54e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:46.239407 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-bbf80375-0d5b-4ebd-893f-cf0d92d8e54e None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:11:46.241361 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-bbf80375-0d5b-4ebd-893f-cf0d92d8e54e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:11:46.275019 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:11:46.285087 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:11:46.310931 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5472e1-17af-45fc-a6e7-09b3c611aa75 None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:46.331100 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 394/790] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 12:11:46 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 15 12:11:46.358672 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a253fc91-e3a5-4abf-af1b-65d2d196ec7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:46.359356 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a253fc91-e3a5-4abf-af1b-65d2d196ec7e None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:46.359647 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a253fc91-e3a5-4abf-af1b-65d2d196ec7e None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:46.359922 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a253fc91-e3a5-4abf-af1b-65d2d196ec7e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:46.372250 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a253fc91-e3a5-4abf-af1b-65d2d196ec7e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:11:46.373720 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a253fc91-e3a5-4abf-af1b-65d2d196ec7e None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:11:46.388697 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application [None req-a253fc91-e3a5-4abf-af1b-65d2d196ec7e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 12:11:46.388697 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:11:46.388697 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:11:46.388697 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:11:46.388697 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:11:46.388697 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:11:46.388697 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:11:46.388697 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:11:46.388697 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:11:46.388697 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:11:46.388697 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:11:46.388697 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:11:46.388697 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 12:11:46.388697 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 12:11:46.388697 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 12:11:46.388697 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 12:11:46.389771 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:11:46.389771 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:11:46.389771 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:11:46.389771 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:11:46.389771 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:11:46.389771 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:11:46.389771 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:11:46.389771 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application with Lock( Sep 15 12:11:46.389771 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:11:46.389771 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:11:46.389771 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:11:46.389771 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:11:46.389771 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:11:46.389771 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:11:46.389771 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:11:46.389771 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:11:46.389771 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 12:11:46.390990 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 12:11:46.390990 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 12:11:46.390990 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application Sep 15 12:11:46.390990 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 397/791] 10.222.0.15 () {56 vars in 1108 bytes} [Thu Sep 15 12:11:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:11:46.397840 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf80375-0d5b-4ebd-893f-cf0d92d8e54e None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:46.547460 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf80375-0d5b-4ebd-893f-cf0d92d8e54e None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:46.595787 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6392879-771f-4da3-803f-477e2091e032 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:46.596405 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6392879-771f-4da3-803f-477e2091e032 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:46.596680 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6392879-771f-4da3-803f-477e2091e032 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:46.596982 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6392879-771f-4da3-803f-477e2091e032 None admin] PATH_INFO: `/v3/domains` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:46.597618 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6392879-771f-4da3-803f-477e2091e032 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:11:46.598463 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6392879-771f-4da3-803f-477e2091e032 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:11:46.599802 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6392879-771f-4da3-803f-477e2091e032 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:11:46.614310 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 395/792] 10.222.0.15 () {56 vars in 1109 bytes} [Thu Sep 15 12:11:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:11:46.619980 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a253fc91-e3a5-4abf-af1b-65d2d196ec7e None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:46.682438 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6392879-771f-4da3-803f-477e2091e032 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:46.748023 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:11:46.797879 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a253fc91-e3a5-4abf-af1b-65d2d196ec7e None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:46.853180 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6392879-771f-4da3-803f-477e2091e032 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:46.873459 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9e16dd-0159-4199-8287-194399b04d70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:46.874152 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9e16dd-0159-4199-8287-194399b04d70 None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:46.874440 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9e16dd-0159-4199-8287-194399b04d70 None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:46.874711 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9e16dd-0159-4199-8287-194399b04d70 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:46.885678 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:11:46.898268 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9e16dd-0159-4199-8287-194399b04d70 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:11:46.900365 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9e16dd-0159-4199-8287-194399b04d70 None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:11:46.914697 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application [None req-0e9e16dd-0159-4199-8287-194399b04d70 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 12:11:46.914697 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:11:46.914697 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:11:46.914697 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:11:46.914697 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:11:46.914697 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:11:46.914697 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:11:46.914697 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:11:46.914697 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:11:46.914697 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:11:46.914697 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:11:46.914697 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:11:46.914697 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 15 12:11:46.914697 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 15 12:11:46.914697 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 15 12:11:46.915782 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 15 12:11:46.915782 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:11:46.915782 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:11:46.915782 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:11:46.915782 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:11:46.915782 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:11:46.915782 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:11:46.915782 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:11:46.915782 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application with Lock( Sep 15 12:11:46.915782 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:11:46.915782 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:11:46.915782 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:11:46.915782 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:11:46.915782 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:11:46.915782 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:11:46.915782 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:11:46.915782 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:11:46.916914 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 15 12:11:46.916914 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 15 12:11:46.916914 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 15 12:11:46.916914 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 15 12:11:46.916914 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 15 12:11:46.916914 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 15 12:11:46.916914 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 12:11:46.916914 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application Sep 15 12:11:46.919043 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 398/793] 10.222.0.15 () {56 vars in 1111 bytes} [Thu Sep 15 12:11:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 299 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:11:46.923113 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46fe987a-4047-4a57-adc9-9f2c2014747b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:46.924029 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fe987a-4047-4a57-adc9-9f2c2014747b None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:46.924315 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fe987a-4047-4a57-adc9-9f2c2014747b None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:46.924592 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fe987a-4047-4a57-adc9-9f2c2014747b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:46.924913 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9e16dd-0159-4199-8287-194399b04d70 None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:46.935629 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46fe987a-4047-4a57-adc9-9f2c2014747b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 12:11:46.944821 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application [None req-46fe987a-4047-4a57-adc9-9f2c2014747b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 15 12:11:46.944821 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:11:46.944821 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:11:46.944821 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:11:46.944821 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:11:46.944821 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:11:46.944821 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:11:46.944821 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:11:46.944821 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:11:46.944821 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:11:46.944821 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:11:46.944821 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:11:46.944821 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 12:11:46.944821 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 12:11:46.944821 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 12:11:46.944821 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 12:11:46.945789 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 12:11:46.945789 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 12:11:46.945789 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 12:11:46.945789 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 12:11:46.945789 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:11:46.945789 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:11:46.945789 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:11:46.945789 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:11:46.945789 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:11:46.945789 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:11:46.945789 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:11:46.945789 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application with Lock( Sep 15 12:11:46.945789 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:11:46.945789 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:11:46.945789 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:11:46.945789 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:11:46.946678 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:11:46.946678 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:11:46.946678 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:11:46.946678 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:11:46.946678 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 12:11:46.946678 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 12:11:46.946678 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 12:11:46.946678 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 12:11:46.946678 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 12:11:46.946678 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 12:11:46.946678 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 15 12:11:46.946678 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 15 12:11:46.946678 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 396/794] 10.222.0.15 () {56 vars in 1096 bytes} [Thu Sep 15 12:11:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 15 12:11:46.948414 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46fe987a-4047-4a57-adc9-9f2c2014747b None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:47.073087 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:11:47.090290 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46fe987a-4047-4a57-adc9-9f2c2014747b None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:47.136613 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023dcb66-7671-45d0-ace7-b430a542338e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:47.137664 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023dcb66-7671-45d0-ace7-b430a542338e None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:47.138972 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023dcb66-7671-45d0-ace7-b430a542338e None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:47.140066 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023dcb66-7671-45d0-ace7-b430a542338e None admin] PATH_INFO: `/v3/roles` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:47.141122 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023dcb66-7671-45d0-ace7-b430a542338e None admin] RBAC: Adding query filter params (name=admin) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:11:47.143186 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9e16dd-0159-4199-8287-194399b04d70 None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:47.143727 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023dcb66-7671-45d0-ace7-b430a542338e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:11:47.145899 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023dcb66-7671-45d0-ace7-b430a542338e None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:11:47.162888 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 397/795] 10.222.0.15 () {56 vars in 1099 bytes} [Thu Sep 15 12:11:46 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:11:47.169746 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023dcb66-7671-45d0-ace7-b430a542338e None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:47.215729 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26790b45-5524-4549-b514-f410908cae20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:47.216389 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26790b45-5524-4549-b514-f410908cae20 None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:47.216670 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26790b45-5524-4549-b514-f410908cae20 None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:47.216965 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26790b45-5524-4549-b514-f410908cae20 None admin] PATH_INFO: `/v3/projects` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:47.217660 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26790b45-5524-4549-b514-f410908cae20 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81449) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:11:47.217962 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26790b45-5524-4549-b514-f410908cae20 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:11:47.219395 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26790b45-5524-4549-b514-f410908cae20 None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:11:47.243506 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 399/796] 10.222.0.15 () {56 vars in 1148 bytes} [Thu Sep 15 12:11:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 319 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 12:11:47.249470 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26790b45-5524-4549-b514-f410908cae20 None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:47.307639 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023dcb66-7671-45d0-ace7-b430a542338e None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:47.320894 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:11:47.347934 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e07d26f5-9fe1-4c30-a4c6-9c0cf92099ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:47.348649 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07d26f5-9fe1-4c30-a4c6-9c0cf92099ab None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:47.349055 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07d26f5-9fe1-4c30-a4c6-9c0cf92099ab None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:47.349424 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e07d26f5-9fe1-4c30-a4c6-9c0cf92099ab None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:47.360935 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07d26f5-9fe1-4c30-a4c6-9c0cf92099ab None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:11:47.362627 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e07d26f5-9fe1-4c30-a4c6-9c0cf92099ab None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:11:47.376433 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26790b45-5524-4549-b514-f410908cae20 None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:47.378762 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application [None req-e07d26f5-9fe1-4c30-a4c6-9c0cf92099ab None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 12:11:47.378762 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:11:47.378762 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:11:47.378762 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:11:47.378762 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:11:47.378762 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:11:47.378762 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:11:47.378762 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:11:47.378762 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:11:47.378762 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:11:47.378762 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:11:47.378762 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:11:47.378762 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 12:11:47.378762 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 12:11:47.378762 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 12:11:47.378762 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 12:11:47.380265 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:11:47.380265 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:11:47.380265 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:11:47.380265 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:11:47.380265 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:11:47.380265 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:11:47.380265 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:11:47.380265 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application with Lock( Sep 15 12:11:47.380265 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:11:47.380265 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:11:47.380265 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:11:47.380265 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:11:47.380265 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:11:47.380265 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:11:47.380265 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:11:47.380265 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:11:47.380265 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 12:11:47.381213 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 12:11:47.381213 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 12:11:47.381213 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 15 12:11:47.385375 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 398/797] 10.222.0.15 () {56 vars in 1108 bytes} [Thu Sep 15 12:11:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:11:47.386729 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b96d2d6-cebf-40ab-bd81-e7e1565157a6 None None] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:47.387091 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b96d2d6-cebf-40ab-bd81-e7e1565157a6 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:47.387443 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b96d2d6-cebf-40ab-bd81-e7e1565157a6 None None] PATH_INFO: `/` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:47.389169 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 399/798] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 12:11:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:11:47.389949 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b96d2d6-cebf-40ab-bd81-e7e1565157a6 None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:47.393784 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:11:47.431383 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b778568f-2619-431f-95c3-ad4f76011f20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:47.433275 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b778568f-2619-431f-95c3-ad4f76011f20 None admin] REQUEST_METHOD: `PUT` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:47.433557 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b778568f-2619-431f-95c3-ad4f76011f20 None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:47.433830 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b778568f-2619-431f-95c3-ad4f76011f20 None admin] PATH_INFO: `/v3/projects/540a32ef88764e8c8d9201ae22f6ddaf/users/efa0d3dc5b7540d59fdd59bbf64a04b6/roles/9c769b5c82884766b2c510b1f0f1364c` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:47.437484 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b778568f-2619-431f-95c3-ad4f76011f20 None admin] RBAC: Authorizing `identity:create_grant(project_id=540a32ef88764e8c8d9201ae22f6ddaf, user_id=efa0d3dc5b7540d59fdd59bbf64a04b6, role_id=9c769b5c82884766b2c510b1f0f1364c)` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:11:47.438925 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b778568f-2619-431f-95c3-ad4f76011f20 None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:11:47.458747 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 400/799] 10.222.0.15 () {58 vars in 1439 bytes} [Thu Sep 15 12:11:47 2022] PUT /identity/v3/projects/540a32ef88764e8c8d9201ae22f6ddaf/users/efa0d3dc5b7540d59fdd59bbf64a04b6/roles/9c769b5c82884766b2c510b1f0f1364c => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:11:47.460273 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f16a9a-57f6-4869-a5ee-e8b90c492b79 None None] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:47.460440 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f16a9a-57f6-4869-a5ee-e8b90c492b79 None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:47.460746 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f16a9a-57f6-4869-a5ee-e8b90c492b79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:47.481595 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.auth.core [None req-c2f16a9a-57f6-4869-a5ee-e8b90c492b79 None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81449) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:11:47.482963 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-c2f16a9a-57f6-4869-a5ee-e8b90c492b79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:11:47.531007 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b96d2d6-cebf-40ab-bd81-e7e1565157a6 None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:47.568580 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 401/800] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 12:11:47 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 15 12:11:47.583360 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb3da51-07b8-4a6d-a3cc-9278914f80c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:47.584709 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb3da51-07b8-4a6d-a3cc-9278914f80c8 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:47.585253 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb3da51-07b8-4a6d-a3cc-9278914f80c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:47.585734 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb3da51-07b8-4a6d-a3cc-9278914f80c8 None admin] PATH_INFO: `/v3/domains` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:47.586675 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb3da51-07b8-4a6d-a3cc-9278914f80c8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:11:47.587173 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb3da51-07b8-4a6d-a3cc-9278914f80c8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:11:47.588778 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb3da51-07b8-4a6d-a3cc-9278914f80c8 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:11:47.602627 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 400/801] 10.222.0.15 () {56 vars in 1109 bytes} [Thu Sep 15 12:11:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:11:47.608660 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f16a9a-57f6-4869-a5ee-e8b90c492b79 None None] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:47.659605 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:11:47.723719 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f16a9a-57f6-4869-a5ee-e8b90c492b79 None None] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:47.761489 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aaafa62-3c4d-4ba2-9b8a-ff087bb0c59f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:47.762447 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aaafa62-3c4d-4ba2-9b8a-ff087bb0c59f None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:47.762942 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aaafa62-3c4d-4ba2-9b8a-ff087bb0c59f None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:47.763408 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aaafa62-3c4d-4ba2-9b8a-ff087bb0c59f None admin] PATH_INFO: `/v3/users/nova` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:47.772543 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aaafa62-3c4d-4ba2-9b8a-ff087bb0c59f None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:11:47.774381 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aaafa62-3c4d-4ba2-9b8a-ff087bb0c59f None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:11:47.787535 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application [None req-3aaafa62-3c4d-4ba2-9b8a-ff087bb0c59f None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 15 12:11:47.787535 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:11:47.787535 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:11:47.787535 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:11:47.787535 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:11:47.787535 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:11:47.787535 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:11:47.787535 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:11:47.787535 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:11:47.787535 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:11:47.787535 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:11:47.787535 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:11:47.787535 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 15 12:11:47.787535 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 15 12:11:47.787535 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 15 12:11:47.787535 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 15 12:11:47.788563 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:11:47.788563 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:11:47.788563 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 15 12:11:47.788563 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 12:11:47.788563 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 15 12:11:47.788563 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 12:11:47.788563 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:11:47.788563 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:11:47.788563 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:11:47.788563 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:11:47.788563 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:11:47.788563 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application with Lock( Sep 15 12:11:47.788563 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:11:47.788563 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:11:47.788563 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:11:47.788563 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:11:47.788563 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:11:47.788563 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:11:47.789561 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:11:47.789561 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:11:47.789561 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 15 12:11:47.789561 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 15 12:11:47.789561 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 15 12:11:47.789561 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 15 12:11:47.789561 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 15 12:11:47.789561 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 15 12:11:47.789561 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 15 12:11:47.789561 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application Sep 15 12:11:47.791858 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 402/802] 10.222.0.15 () {56 vars in 1093 bytes} [Thu Sep 15 12:11:47 2022] GET /identity/v3/users/nova => generated 82 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:11:47.799173 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb3da51-07b8-4a6d-a3cc-9278914f80c8 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:47.860063 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aaafa62-3c4d-4ba2-9b8a-ff087bb0c59f None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:47.870032 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:11:47.914074 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb3da51-07b8-4a6d-a3cc-9278914f80c8 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:47.956847 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f1700b-137a-46ca-a6ee-dc84f630acdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:47.958037 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f1700b-137a-46ca-a6ee-dc84f630acdb None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:47.958312 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f1700b-137a-46ca-a6ee-dc84f630acdb None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:47.958580 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f1700b-137a-46ca-a6ee-dc84f630acdb None admin] PATH_INFO: `/v3/users` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:47.959307 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f1700b-137a-46ca-a6ee-dc84f630acdb None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:11:47.959605 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f1700b-137a-46ca-a6ee-dc84f630acdb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:11:47.961910 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f1700b-137a-46ca-a6ee-dc84f630acdb None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:11:47.986757 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 401/803] 10.222.0.15 () {56 vars in 1133 bytes} [Thu Sep 15 12:11:47 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 12:11:47.988501 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb4e99f-ff63-4891-aabf-6ed4220d3cc6 None None] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:47.988983 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb4e99f-ff63-4891-aabf-6ed4220d3cc6 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:47.989397 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb4e99f-ff63-4891-aabf-6ed4220d3cc6 None None] PATH_INFO: `/` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:47.995221 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 402/804] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 12:11:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:11:47.996264 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb4e99f-ff63-4891-aabf-6ed4220d3cc6 None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:48.032177 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aaafa62-3c4d-4ba2-9b8a-ff087bb0c59f None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:48.035973 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:11:48.080817 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4fadc2b-9687-4949-9bf7-38e1f76be4e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:48.081493 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fadc2b-9687-4949-9bf7-38e1f76be4e8 None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:48.081777 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fadc2b-9687-4949-9bf7-38e1f76be4e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:48.082106 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4fadc2b-9687-4949-9bf7-38e1f76be4e8 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:48.083565 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4fadc2b-9687-4949-9bf7-38e1f76be4e8 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:11:48.084985 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4fadc2b-9687-4949-9bf7-38e1f76be4e8 None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:11:48.097219 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application [None req-c4fadc2b-9687-4949-9bf7-38e1f76be4e8 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 15 12:11:48.097219 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:11:48.097219 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 15 12:11:48.097219 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 15 12:11:48.097219 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 15 12:11:48.097219 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 15 12:11:48.097219 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 15 12:11:48.097219 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 15 12:11:48.097219 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 15 12:11:48.097219 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application Sep 15 12:11:48.097219 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 15 12:11:48.097219 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application Sep 15 12:11:48.097219 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:11:48.097219 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:11:48.097219 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:11:48.097219 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:11:48.097219 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:11:48.098815 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:11:48.098815 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:11:48.098815 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:11:48.098815 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:11:48.098815 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:11:48.098815 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:11:48.098815 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 15 12:11:48.098815 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 15 12:11:48.098815 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 15 12:11:48.098815 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 15 12:11:48.098815 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:11:48.098815 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:11:48.098815 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:11:48.098815 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:11:48.098815 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:11:48.098815 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:11:48.099940 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:11:48.099940 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application with Lock( Sep 15 12:11:48.099940 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:11:48.099940 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:11:48.099940 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:11:48.099940 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:11:48.099940 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:11:48.099940 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:11:48.099940 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:11:48.099940 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:11:48.099940 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 15 12:11:48.099940 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 15 12:11:48.099940 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Sep 15 12:11:48.099940 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application Sep 15 12:11:48.103963 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 403/805] 10.222.0.15 () {56 vars in 1129 bytes} [Thu Sep 15 12:11:47 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 15 12:11:48.106120 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa462275-3de7-4b4d-96fa-1f45d4682429 None None] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:48.106412 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa462275-3de7-4b4d-96fa-1f45d4682429 None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:48.106687 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa462275-3de7-4b4d-96fa-1f45d4682429 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:48.124692 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb4e99f-ff63-4891-aabf-6ed4220d3cc6 None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:48.130650 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.auth.core [None req-fa462275-3de7-4b4d-96fa-1f45d4682429 None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81449) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:11:48.131874 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-fa462275-3de7-4b4d-96fa-1f45d4682429 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:11:48.148012 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:11:48.165717 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d564337d-22be-469d-9e01-68770320eb83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:48.166885 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d564337d-22be-469d-9e01-68770320eb83 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:48.167424 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d564337d-22be-469d-9e01-68770320eb83 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:48.167860 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d564337d-22be-469d-9e01-68770320eb83 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:48.228802 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 404/806] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 12:11:48 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 15 12:11:48.228802 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa462275-3de7-4b4d-96fa-1f45d4682429 None None] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:48.275176 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d564337d-22be-469d-9e01-68770320eb83 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:11:48.276623 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d564337d-22be-469d-9e01-68770320eb83 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:11:48.413317 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 403/807] 10.222.0.15 () {58 vars in 1282 bytes} [Thu Sep 15 12:11:47 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 417 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 15 12:11:48.421884 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d564337d-22be-469d-9e01-68770320eb83 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:48.434486 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:11:48.439861 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa462275-3de7-4b4d-96fa-1f45d4682429 None None] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:48.465085 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:11:48.483754 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c46dad0-44ce-475b-9b17-cb8524fd9ce3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:48.484598 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c46dad0-44ce-475b-9b17-cb8524fd9ce3 None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:48.485068 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c46dad0-44ce-475b-9b17-cb8524fd9ce3 None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:48.485488 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c46dad0-44ce-475b-9b17-cb8524fd9ce3 None admin] PATH_INFO: `/v3/services` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:48.486263 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c46dad0-44ce-475b-9b17-cb8524fd9ce3 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81449) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:11:48.486817 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c46dad0-44ce-475b-9b17-cb8524fd9ce3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:11:48.488364 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c46dad0-44ce-475b-9b17-cb8524fd9ce3 None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:11:48.493878 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 405/808] 10.222.0.15 () {56 vars in 1124 bytes} [Thu Sep 15 12:11:48 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:11:48.495386 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19916212-d8dd-4e20-88a0-e37632ef238a None None] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:48.495885 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19916212-d8dd-4e20-88a0-e37632ef238a None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:48.496400 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19916212-d8dd-4e20-88a0-e37632ef238a None None] PATH_INFO: `/` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:48.498517 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 406/809] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 12:11:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:11:48.499428 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19916212-d8dd-4e20-88a0-e37632ef238a None None] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:48.555602 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d564337d-22be-469d-9e01-68770320eb83 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:48.592498 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86f565a-7ec9-4b28-bc12-f1ae2cf78559 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:48.593193 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86f565a-7ec9-4b28-bc12-f1ae2cf78559 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:48.593447 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86f565a-7ec9-4b28-bc12-f1ae2cf78559 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:48.593719 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86f565a-7ec9-4b28-bc12-f1ae2cf78559 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:48.604978 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86f565a-7ec9-4b28-bc12-f1ae2cf78559 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:11:48.606440 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86f565a-7ec9-4b28-bc12-f1ae2cf78559 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:11:48.621737 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application [None req-f86f565a-7ec9-4b28-bc12-f1ae2cf78559 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 12:11:48.621737 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:11:48.621737 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:11:48.621737 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:11:48.621737 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:11:48.621737 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:11:48.621737 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:11:48.621737 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:11:48.621737 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:11:48.621737 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:11:48.621737 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:11:48.621737 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:11:48.621737 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 12:11:48.621737 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 12:11:48.621737 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 12:11:48.621737 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 12:11:48.622745 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:11:48.622745 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:11:48.622745 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:11:48.622745 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:11:48.622745 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:11:48.622745 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:11:48.622745 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:11:48.622745 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application with Lock( Sep 15 12:11:48.622745 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:11:48.622745 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:11:48.622745 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:11:48.622745 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:11:48.622745 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:11:48.622745 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:11:48.622745 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:11:48.622745 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:11:48.622745 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 12:11:48.624470 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 12:11:48.624470 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 12:11:48.624470 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 15 12:11:48.624470 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 404/810] 10.222.0.15 () {56 vars in 1108 bytes} [Thu Sep 15 12:11:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:11:48.624470 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f505f6e8-5217-4e1d-b1b2-edae5d94c4ea None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:48.624470 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f505f6e8-5217-4e1d-b1b2-edae5d94c4ea None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:48.624928 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f505f6e8-5217-4e1d-b1b2-edae5d94c4ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:48.645740 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-f505f6e8-5217-4e1d-b1b2-edae5d94c4ea None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:11:48.646934 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-f505f6e8-5217-4e1d-b1b2-edae5d94c4ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:11:48.718502 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 405/811] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 12:11:48 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 15 12:11:48.719351 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f505f6e8-5217-4e1d-b1b2-edae5d94c4ea None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:48.721672 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:11:48.759206 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19916212-d8dd-4e20-88a0-e37632ef238a None None] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:48.807505 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490cadba-769b-4793-ac88-3d5f39353250 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:48.808886 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490cadba-769b-4793-ac88-3d5f39353250 None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:48.809463 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490cadba-769b-4793-ac88-3d5f39353250 None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:48.809925 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490cadba-769b-4793-ac88-3d5f39353250 None admin] PATH_INFO: `/v3/services` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:48.810964 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-490cadba-769b-4793-ac88-3d5f39353250 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81449) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:11:48.811463 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-490cadba-769b-4793-ac88-3d5f39353250 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:11:48.813691 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-490cadba-769b-4793-ac88-3d5f39353250 None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:11:48.821008 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 407/812] 10.222.0.15 () {56 vars in 1124 bytes} [Thu Sep 15 12:11:48 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 322 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:11:48.822707 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490cadba-769b-4793-ac88-3d5f39353250 None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:48.834248 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f505f6e8-5217-4e1d-b1b2-edae5d94c4ea None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:48.865219 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:11:48.874000 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164a2158-58bb-474f-8b56-ca850166e412 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:48.874720 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164a2158-58bb-474f-8b56-ca850166e412 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:48.875408 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164a2158-58bb-474f-8b56-ca850166e412 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:48.875685 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164a2158-58bb-474f-8b56-ca850166e412 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:48.889685 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164a2158-58bb-474f-8b56-ca850166e412 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 12:11:48.895803 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application [None req-164a2158-58bb-474f-8b56-ca850166e412 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 15 12:11:48.895803 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:11:48.895803 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:11:48.895803 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:11:48.895803 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:11:48.895803 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:11:48.895803 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:11:48.895803 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:11:48.895803 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:11:48.895803 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:11:48.895803 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:11:48.895803 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:11:48.895803 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 12:11:48.895803 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 12:11:48.895803 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 12:11:48.895803 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 12:11:48.896799 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 12:11:48.896799 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 12:11:48.896799 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 12:11:48.896799 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 12:11:48.896799 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:11:48.896799 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:11:48.896799 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:11:48.896799 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:11:48.896799 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:11:48.896799 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:11:48.896799 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:11:48.896799 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application with Lock( Sep 15 12:11:48.896799 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:11:48.896799 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:11:48.896799 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:11:48.896799 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:11:48.897840 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:11:48.897840 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:11:48.897840 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:11:48.897840 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:11:48.897840 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 12:11:48.897840 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 12:11:48.897840 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 12:11:48.897840 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 12:11:48.897840 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 12:11:48.897840 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 12:11:48.897840 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 15 12:11:48.897840 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 15 12:11:48.900105 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 406/813] 10.222.0.15 () {56 vars in 1102 bytes} [Thu Sep 15 12:11:48 2022] GET /identity/v3/roles/service => generated 85 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:11:48.901023 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164a2158-58bb-474f-8b56-ca850166e412 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:48.950802 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490cadba-769b-4793-ac88-3d5f39353250 None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:48.989443 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:11:49.007692 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee57f45-4bba-433b-b957-d801bef74229 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:49.009008 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee57f45-4bba-433b-b957-d801bef74229 None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:49.009594 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee57f45-4bba-433b-b957-d801bef74229 None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:49.010123 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee57f45-4bba-433b-b957-d801bef74229 None admin] PATH_INFO: `/v3/domains` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:49.010776 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee57f45-4bba-433b-b957-d801bef74229 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81449) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:11:49.011078 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee57f45-4bba-433b-b957-d801bef74229 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:11:49.013095 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee57f45-4bba-433b-b957-d801bef74229 None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:11:49.025592 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 408/814] 10.222.0.15 () {56 vars in 1109 bytes} [Thu Sep 15 12:11:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:11:49.028939 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee57f45-4bba-433b-b957-d801bef74229 None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:49.034360 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164a2158-58bb-474f-8b56-ca850166e412 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:49.075407 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0300de3a-7cee-42cd-b05b-6e55bbf07273 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:49.076059 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0300de3a-7cee-42cd-b05b-6e55bbf07273 None admin] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:49.076301 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0300de3a-7cee-42cd-b05b-6e55bbf07273 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:49.076570 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0300de3a-7cee-42cd-b05b-6e55bbf07273 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:49.077253 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0300de3a-7cee-42cd-b05b-6e55bbf07273 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:11:49.078613 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0300de3a-7cee-42cd-b05b-6e55bbf07273 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:11:49.102690 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 407/815] 10.222.0.15 () {60 vars in 1144 bytes} [Thu Sep 15 12:11:48 2022] POST /identity/v3/endpoints => generated 346 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 12:11:49.103823 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0300de3a-7cee-42cd-b05b-6e55bbf07273 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:49.119182 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:11:49.145692 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee57f45-4bba-433b-b957-d801bef74229 None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:49.193061 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12fd342b-19b8-4270-8ccc-e7402f6e6014 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:49.193768 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fd342b-19b8-4270-8ccc-e7402f6e6014 None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:49.194058 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fd342b-19b8-4270-8ccc-e7402f6e6014 None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:49.194322 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fd342b-19b8-4270-8ccc-e7402f6e6014 None admin] PATH_INFO: `/v3/roles` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:49.195000 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12fd342b-19b8-4270-8ccc-e7402f6e6014 None admin] RBAC: Adding query filter params (name=service) {{(pid=81449) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:11:49.195299 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12fd342b-19b8-4270-8ccc-e7402f6e6014 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:11:49.196786 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12fd342b-19b8-4270-8ccc-e7402f6e6014 None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:11:49.207111 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 409/816] 10.222.0.15 () {56 vars in 1103 bytes} [Thu Sep 15 12:11:49 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:11:49.208084 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12fd342b-19b8-4270-8ccc-e7402f6e6014 None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:49.287394 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0300de3a-7cee-42cd-b05b-6e55bbf07273 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:49.312576 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:11:49.325443 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12fd342b-19b8-4270-8ccc-e7402f6e6014 None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:49.335099 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2b8b3d-1fb7-4ad3-8725-016ff4cc7088 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:49.335975 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2b8b3d-1fb7-4ad3-8725-016ff4cc7088 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:49.336515 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2b8b3d-1fb7-4ad3-8725-016ff4cc7088 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:49.336515 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2b8b3d-1fb7-4ad3-8725-016ff4cc7088 None admin] PATH_INFO: `/v3/services/network` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:49.338662 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2b8b3d-1fb7-4ad3-8725-016ff4cc7088 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:11:49.338662 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2b8b3d-1fb7-4ad3-8725-016ff4cc7088 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:11:49.352033 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application [None req-4a2b8b3d-1fb7-4ad3-8725-016ff4cc7088 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Sep 15 12:11:49.352033 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:11:49.352033 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 15 12:11:49.352033 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 15 12:11:49.352033 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 15 12:11:49.352033 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 15 12:11:49.352033 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 15 12:11:49.352033 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 15 12:11:49.352033 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 15 12:11:49.352033 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 15 12:11:49.352033 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 15 12:11:49.352033 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 15 12:11:49.352033 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:11:49.352033 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:11:49.352033 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:11:49.352033 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:11:49.352033 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:11:49.353231 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:11:49.353231 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:11:49.353231 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:11:49.353231 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:11:49.353231 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:11:49.353231 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:11:49.353231 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 15 12:11:49.353231 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 15 12:11:49.353231 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 15 12:11:49.353231 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 15 12:11:49.353231 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:11:49.353231 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:11:49.353231 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:11:49.353231 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:11:49.353231 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:11:49.353231 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:11:49.354080 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:11:49.354080 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application with Lock( Sep 15 12:11:49.354080 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:11:49.354080 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:11:49.354080 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:11:49.354080 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:11:49.354080 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:11:49.354080 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:11:49.354080 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:11:49.354080 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:11:49.354080 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 15 12:11:49.354080 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 15 12:11:49.354080 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Sep 15 12:11:49.354080 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 15 12:11:49.354080 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 408/817] 10.222.0.15 () {56 vars in 1111 bytes} [Thu Sep 15 12:11:49 2022] GET /identity/v3/services/network => generated 88 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:11:49.355176 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2b8b3d-1fb7-4ad3-8725-016ff4cc7088 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:49.376310 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c0828f-ed44-4fba-bb5a-9da80475c13f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:49.377260 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c0828f-ed44-4fba-bb5a-9da80475c13f None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:49.377658 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c0828f-ed44-4fba-bb5a-9da80475c13f None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:49.378032 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c0828f-ed44-4fba-bb5a-9da80475c13f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:49.385904 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c0828f-ed44-4fba-bb5a-9da80475c13f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:11:49.387586 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c0828f-ed44-4fba-bb5a-9da80475c13f None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:11:49.400094 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application [None req-d8c0828f-ed44-4fba-bb5a-9da80475c13f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 12:11:49.400094 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:11:49.400094 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:11:49.400094 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:11:49.400094 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:11:49.400094 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:11:49.400094 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:11:49.400094 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:11:49.400094 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:11:49.400094 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:11:49.400094 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:11:49.400094 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:11:49.400094 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 15 12:11:49.400094 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 15 12:11:49.400094 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 15 12:11:49.401247 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 15 12:11:49.401247 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:11:49.401247 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:11:49.401247 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:11:49.401247 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:11:49.401247 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:11:49.401247 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:11:49.401247 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:11:49.401247 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application with Lock( Sep 15 12:11:49.401247 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:11:49.401247 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:11:49.401247 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:11:49.401247 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:11:49.401247 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:11:49.401247 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:11:49.401247 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:11:49.401247 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:11:49.402386 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 15 12:11:49.402386 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 15 12:11:49.402386 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 15 12:11:49.402386 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 15 12:11:49.402386 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 15 12:11:49.402386 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 15 12:11:49.402386 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 12:11:49.402386 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application Sep 15 12:11:49.404787 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 410/818] 10.222.0.15 () {56 vars in 1111 bytes} [Thu Sep 15 12:11:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:11:49.404787 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c0828f-ed44-4fba-bb5a-9da80475c13f None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:49.413376 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:11:49.494093 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2b8b3d-1fb7-4ad3-8725-016ff4cc7088 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:49.532296 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5be749-60f4-4c05-9caa-101e18acf278 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:49.533362 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5be749-60f4-4c05-9caa-101e18acf278 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:49.534109 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5be749-60f4-4c05-9caa-101e18acf278 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:49.534713 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5be749-60f4-4c05-9caa-101e18acf278 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:49.545245 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c0828f-ed44-4fba-bb5a-9da80475c13f None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:49.546965 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c5be749-60f4-4c05-9caa-101e18acf278 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:11:49.548547 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c5be749-60f4-4c05-9caa-101e18acf278 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:11:49.565441 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application [None req-9c5be749-60f4-4c05-9caa-101e18acf278 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 12:11:49.565441 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:11:49.565441 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:11:49.565441 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:11:49.565441 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:11:49.565441 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:11:49.565441 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:11:49.565441 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:11:49.565441 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:11:49.565441 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:11:49.565441 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:11:49.565441 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:11:49.565441 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 12:11:49.565441 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 12:11:49.565441 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 12:11:49.565441 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 12:11:49.566464 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:11:49.566464 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:11:49.566464 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:11:49.566464 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:11:49.566464 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:11:49.566464 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:11:49.566464 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:11:49.566464 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application with Lock( Sep 15 12:11:49.566464 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:11:49.566464 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:11:49.566464 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:11:49.566464 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:11:49.566464 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:11:49.566464 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:11:49.566464 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:11:49.566464 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:11:49.566464 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 12:11:49.567438 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 12:11:49.567438 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 12:11:49.567438 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 15 12:11:49.573876 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 409/819] 10.222.0.15 () {56 vars in 1108 bytes} [Thu Sep 15 12:11:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:11:49.573876 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5be749-60f4-4c05-9caa-101e18acf278 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:49.591607 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085456a9-fdf7-4d6b-a014-6a0029aaa03f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:49.593052 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085456a9-fdf7-4d6b-a014-6a0029aaa03f None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:49.593749 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085456a9-fdf7-4d6b-a014-6a0029aaa03f None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:49.594234 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085456a9-fdf7-4d6b-a014-6a0029aaa03f None admin] PATH_INFO: `/v3/services` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:49.595401 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085456a9-fdf7-4d6b-a014-6a0029aaa03f None admin] RBAC: Adding query filter params (name=network) {{(pid=81449) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:11:49.595921 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085456a9-fdf7-4d6b-a014-6a0029aaa03f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:11:49.598801 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085456a9-fdf7-4d6b-a014-6a0029aaa03f None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:11:49.605703 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 411/820] 10.222.0.15 () {56 vars in 1112 bytes} [Thu Sep 15 12:11:49 2022] GET /identity/v3/services?name=network => generated 124 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:11:49.606882 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085456a9-fdf7-4d6b-a014-6a0029aaa03f None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:49.618412 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:11:49.701135 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:11:49.701801 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5be749-60f4-4c05-9caa-101e18acf278 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:49.743538 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ac3407-7a4f-4125-a1af-f56c664988b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:49.743850 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085456a9-fdf7-4d6b-a014-6a0029aaa03f None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:49.744638 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ac3407-7a4f-4125-a1af-f56c664988b1 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:49.745272 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ac3407-7a4f-4125-a1af-f56c664988b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:49.745814 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ac3407-7a4f-4125-a1af-f56c664988b1 None admin] PATH_INFO: `/v3/projects` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:49.746798 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ac3407-7a4f-4125-a1af-f56c664988b1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:11:49.747351 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ac3407-7a4f-4125-a1af-f56c664988b1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:11:49.749035 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ac3407-7a4f-4125-a1af-f56c664988b1 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:11:49.763021 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 410/821] 10.222.0.15 () {56 vars in 1148 bytes} [Thu Sep 15 12:11:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:11:49.764435 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ac3407-7a4f-4125-a1af-f56c664988b1 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:49.789917 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30e17cb-7743-4b6d-8900-c31ec40fe73f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:49.790535 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30e17cb-7743-4b6d-8900-c31ec40fe73f None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:49.790812 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30e17cb-7743-4b6d-8900-c31ec40fe73f None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:49.791260 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30e17cb-7743-4b6d-8900-c31ec40fe73f None admin] PATH_INFO: `/v3/domains` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:49.792353 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30e17cb-7743-4b6d-8900-c31ec40fe73f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81449) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:11:49.792665 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30e17cb-7743-4b6d-8900-c31ec40fe73f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:11:49.794362 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30e17cb-7743-4b6d-8900-c31ec40fe73f None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:11:49.816281 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 412/822] 10.222.0.15 () {56 vars in 1109 bytes} [Thu Sep 15 12:11:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:11:49.816281 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30e17cb-7743-4b6d-8900-c31ec40fe73f None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:49.876018 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:11:49.886182 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ac3407-7a4f-4125-a1af-f56c664988b1 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:49.931125 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67282b87-98ba-4350-831b-7c1fc3fd23b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:49.931901 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67282b87-98ba-4350-831b-7c1fc3fd23b6 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:49.932283 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67282b87-98ba-4350-831b-7c1fc3fd23b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:49.932658 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67282b87-98ba-4350-831b-7c1fc3fd23b6 None admin] PATH_INFO: `/v3/services` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:49.933433 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67282b87-98ba-4350-831b-7c1fc3fd23b6 None admin] RBAC: Adding query filter params (type=network) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:11:49.933611 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30e17cb-7743-4b6d-8900-c31ec40fe73f None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:49.934057 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67282b87-98ba-4350-831b-7c1fc3fd23b6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:11:49.935372 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67282b87-98ba-4350-831b-7c1fc3fd23b6 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:11:49.946476 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 411/823] 10.222.0.15 () {56 vars in 1112 bytes} [Thu Sep 15 12:11:49 2022] GET /identity/v3/services?type=network => generated 350 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:11:49.947721 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67282b87-98ba-4350-831b-7c1fc3fd23b6 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:49.969571 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:11:49.985754 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c266f89-6d5e-4781-ac47-d3101d2393f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:49.986684 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c266f89-6d5e-4781-ac47-d3101d2393f9 None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:49.987120 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c266f89-6d5e-4781-ac47-d3101d2393f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:49.987513 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c266f89-6d5e-4781-ac47-d3101d2393f9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:49.988968 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c266f89-6d5e-4781-ac47-d3101d2393f9 None admin] RBAC: Adding query filter params (role.id=9c769b5c82884766b2c510b1f0f1364c, scope.project.id=540a32ef88764e8c8d9201ae22f6ddaf, user.id=eface89e261840a2abb4206be26455c7) {{(pid=81449) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:11:49.989432 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c266f89-6d5e-4781-ac47-d3101d2393f9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:11:49.991044 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c266f89-6d5e-4781-ac47-d3101d2393f9 None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:11:49.998276 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 413/824] 10.222.0.15 () {56 vars in 1374 bytes} [Thu Sep 15 12:11:49 2022] GET /identity/v3/role_assignments?user.id=eface89e261840a2abb4206be26455c7&scope.project.id=540a32ef88764e8c8d9201ae22f6ddaf&role.id=9c769b5c82884766b2c510b1f0f1364c => generated 608 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:11:49.999291 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c266f89-6d5e-4781-ac47-d3101d2393f9 None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:50.098278 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67282b87-98ba-4350-831b-7c1fc3fd23b6 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:50.124456 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c266f89-6d5e-4781-ac47-d3101d2393f9 None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:50.149012 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3ed5b0-6568-40ba-b7b3-f999429d6751 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:50.149824 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3ed5b0-6568-40ba-b7b3-f999429d6751 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:50.150584 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3ed5b0-6568-40ba-b7b3-f999429d6751 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:50.150584 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3ed5b0-6568-40ba-b7b3-f999429d6751 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:50.159331 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e3ed5b0-6568-40ba-b7b3-f999429d6751 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:11:50.161053 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e3ed5b0-6568-40ba-b7b3-f999429d6751 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:11:50.169023 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f7a77a-12cb-4b90-9147-c1dfea3b8937 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:50.169889 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f7a77a-12cb-4b90-9147-c1dfea3b8937 None admin] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:50.170311 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f7a77a-12cb-4b90-9147-c1dfea3b8937 None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:50.170720 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f7a77a-12cb-4b90-9147-c1dfea3b8937 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:50.171525 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f7a77a-12cb-4b90-9147-c1dfea3b8937 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:11:50.173088 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f7a77a-12cb-4b90-9147-c1dfea3b8937 None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:11:50.174422 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application [None req-6e3ed5b0-6568-40ba-b7b3-f999429d6751 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 15 12:11:50.174422 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:11:50.174422 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:11:50.174422 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:11:50.174422 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:11:50.174422 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:11:50.174422 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:11:50.174422 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:11:50.174422 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:11:50.174422 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:11:50.174422 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:11:50.174422 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:11:50.174422 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 15 12:11:50.174422 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 15 12:11:50.174422 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 15 12:11:50.174422 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 15 12:11:50.175479 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:11:50.175479 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:11:50.175479 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 15 12:11:50.175479 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 12:11:50.175479 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 15 12:11:50.175479 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 12:11:50.175479 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:11:50.175479 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:11:50.175479 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:11:50.175479 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:11:50.175479 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:11:50.175479 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application with Lock( Sep 15 12:11:50.175479 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:11:50.175479 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:11:50.175479 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:11:50.175479 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:11:50.175479 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:11:50.175479 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:11:50.176457 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:11:50.176457 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:11:50.176457 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 15 12:11:50.176457 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 15 12:11:50.176457 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 15 12:11:50.176457 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 15 12:11:50.176457 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 15 12:11:50.176457 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 15 12:11:50.176457 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 15 12:11:50.176457 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 15 12:11:50.176457 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 412/825] 10.222.0.15 () {56 vars in 1117 bytes} [Thu Sep 15 12:11:49 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:11:50.178404 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5562aa6c-1db9-4766-abdc-5c37a021d204 None None] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:50.178785 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5562aa6c-1db9-4766-abdc-5c37a021d204 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:50.179146 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5562aa6c-1db9-4766-abdc-5c37a021d204 None None] PATH_INFO: `/` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:50.185047 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 413/826] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 12:11:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:11:50.185882 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5562aa6c-1db9-4766-abdc-5c37a021d204 None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:50.202777 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 414/827] 10.222.0.15 () {60 vars in 1144 bytes} [Thu Sep 15 12:11:49 2022] POST /identity/v3/endpoints => generated 337 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 12:11:50.204393 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40933b46-b0cb-44be-860c-6df56150c383 None None] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:50.205133 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40933b46-b0cb-44be-860c-6df56150c383 None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:50.205436 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40933b46-b0cb-44be-860c-6df56150c383 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:50.209079 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:11:50.230492 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.auth.core [None req-40933b46-b0cb-44be-860c-6df56150c383 None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81449) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:11:50.231166 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-40933b46-b0cb-44be-860c-6df56150c383 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:11:50.291876 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:11:50.332791 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5562aa6c-1db9-4766-abdc-5c37a021d204 None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:50.333841 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 415/828] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 12:11:50 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 15 12:11:50.391216 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e253a5-1175-44ae-80b1-d9ebbc04a186 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:50.391767 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e253a5-1175-44ae-80b1-d9ebbc04a186 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:50.392051 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e253a5-1175-44ae-80b1-d9ebbc04a186 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:50.392324 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e253a5-1175-44ae-80b1-d9ebbc04a186 None admin] PATH_INFO: `/v3/users` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:50.393105 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e253a5-1175-44ae-80b1-d9ebbc04a186 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:11:50.393411 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e253a5-1175-44ae-80b1-d9ebbc04a186 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:11:50.395417 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07e253a5-1175-44ae-80b1-d9ebbc04a186 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:11:50.414675 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 414/829] 10.222.0.15 () {56 vars in 1149 bytes} [Thu Sep 15 12:11:50 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:11:50.420237 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40933b46-b0cb-44be-860c-6df56150c383 None None] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:50.527138 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40933b46-b0cb-44be-860c-6df56150c383 None None] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:50.562981 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4640c7-01f2-4d88-9095-5f3ef730ae28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:50.563613 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4640c7-01f2-4d88-9095-5f3ef730ae28 None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:50.563893 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4640c7-01f2-4d88-9095-5f3ef730ae28 None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:50.564150 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4640c7-01f2-4d88-9095-5f3ef730ae28 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:50.575778 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4640c7-01f2-4d88-9095-5f3ef730ae28 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:11:50.577456 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4640c7-01f2-4d88-9095-5f3ef730ae28 None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:11:50.595258 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application [None req-bb4640c7-01f2-4d88-9095-5f3ef730ae28 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 12:11:50.595258 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:11:50.595258 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:11:50.595258 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:11:50.595258 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:11:50.595258 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:11:50.595258 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:11:50.595258 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:11:50.595258 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:11:50.595258 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:11:50.595258 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:11:50.595258 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:11:50.595258 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 12:11:50.595258 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 12:11:50.595258 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 12:11:50.595258 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 12:11:50.596258 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:11:50.596258 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:11:50.596258 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:11:50.596258 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:11:50.596258 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:11:50.596258 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:11:50.596258 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:11:50.596258 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application with Lock( Sep 15 12:11:50.596258 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:11:50.596258 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:11:50.596258 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:11:50.596258 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:11:50.596258 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:11:50.596258 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:11:50.596258 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:11:50.596258 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:11:50.596258 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 12:11:50.597234 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 12:11:50.597234 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 12:11:50.597234 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application Sep 15 12:11:50.597234 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 416/830] 10.222.0.15 () {56 vars in 1108 bytes} [Thu Sep 15 12:11:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:11:50.601894 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e253a5-1175-44ae-80b1-d9ebbc04a186 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:50.638954 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4640c7-01f2-4d88-9095-5f3ef730ae28 None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:50.685180 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:11:50.696529 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:11:50.733642 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e253a5-1175-44ae-80b1-d9ebbc04a186 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:50.768571 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4640c7-01f2-4d88-9095-5f3ef730ae28 None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:50.781231 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58936fd9-70a8-48d1-ad6a-2f14493cb7f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:50.782183 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58936fd9-70a8-48d1-ad6a-2f14493cb7f7 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:50.782476 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58936fd9-70a8-48d1-ad6a-2f14493cb7f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:50.783035 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58936fd9-70a8-48d1-ad6a-2f14493cb7f7 None admin] PATH_INFO: `/v3/domains` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:50.783689 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58936fd9-70a8-48d1-ad6a-2f14493cb7f7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:11:50.784000 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58936fd9-70a8-48d1-ad6a-2f14493cb7f7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:11:50.785997 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58936fd9-70a8-48d1-ad6a-2f14493cb7f7 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:11:50.799886 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 415/831] 10.222.0.15 () {56 vars in 1109 bytes} [Thu Sep 15 12:11:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 12:11:50.805546 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58936fd9-70a8-48d1-ad6a-2f14493cb7f7 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:50.813803 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa172fc-bbde-4043-bb01-74ba37ce91cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:50.814538 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa172fc-bbde-4043-bb01-74ba37ce91cb None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:50.814882 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa172fc-bbde-4043-bb01-74ba37ce91cb None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:50.815215 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa172fc-bbde-4043-bb01-74ba37ce91cb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:50.826624 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa172fc-bbde-4043-bb01-74ba37ce91cb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 12:11:50.831512 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application [None req-2fa172fc-bbde-4043-bb01-74ba37ce91cb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 15 12:11:50.831512 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:11:50.831512 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:11:50.831512 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:11:50.831512 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:11:50.831512 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:11:50.831512 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:11:50.831512 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:11:50.831512 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:11:50.831512 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:11:50.831512 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:11:50.831512 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:11:50.831512 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 12:11:50.831512 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 12:11:50.831512 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 12:11:50.831512 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 12:11:50.833115 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 12:11:50.833115 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 12:11:50.833115 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 12:11:50.833115 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 12:11:50.833115 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:11:50.833115 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:11:50.833115 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:11:50.833115 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:11:50.833115 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:11:50.833115 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:11:50.833115 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:11:50.833115 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application with Lock( Sep 15 12:11:50.833115 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:11:50.833115 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:11:50.833115 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:11:50.833115 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:11:50.833954 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:11:50.833954 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:11:50.833954 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:11:50.833954 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:11:50.833954 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 12:11:50.833954 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 12:11:50.833954 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 12:11:50.833954 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 12:11:50.833954 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 12:11:50.833954 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 12:11:50.833954 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 15 12:11:50.833954 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application Sep 15 12:11:50.833954 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 417/832] 10.222.0.15 () {56 vars in 1096 bytes} [Thu Sep 15 12:11:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 15 12:11:50.841120 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa172fc-bbde-4043-bb01-74ba37ce91cb None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:50.919072 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58936fd9-70a8-48d1-ad6a-2f14493cb7f7 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:50.952097 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:11:50.957717 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:11:50.965317 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206a34d3-62d1-401d-af13-f15e5d3be727 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:50.966000 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206a34d3-62d1-401d-af13-f15e5d3be727 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:50.968224 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206a34d3-62d1-401d-af13-f15e5d3be727 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:50.968499 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206a34d3-62d1-401d-af13-f15e5d3be727 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:50.980241 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-206a34d3-62d1-401d-af13-f15e5d3be727 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:11:50.981737 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-206a34d3-62d1-401d-af13-f15e5d3be727 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:11:50.983673 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa172fc-bbde-4043-bb01-74ba37ce91cb None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:50.993710 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application [None req-206a34d3-62d1-401d-af13-f15e5d3be727 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 12:11:50.993710 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:11:50.993710 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:11:50.993710 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:11:50.993710 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:11:50.993710 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:11:50.993710 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:11:50.993710 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:11:50.993710 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:11:50.993710 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:11:50.993710 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:11:50.993710 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:11:50.993710 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 15 12:11:50.993710 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 15 12:11:50.993710 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 15 12:11:50.994633 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 15 12:11:50.994633 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:11:50.994633 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:11:50.994633 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:11:50.994633 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:11:50.994633 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:11:50.994633 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:11:50.994633 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:11:50.994633 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application with Lock( Sep 15 12:11:50.994633 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:11:50.994633 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:11:50.994633 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:11:50.994633 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:11:50.994633 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:11:50.994633 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:11:50.994633 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:11:50.994633 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:11:50.995879 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 15 12:11:50.995879 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 15 12:11:50.995879 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 15 12:11:50.995879 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 15 12:11:50.995879 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 15 12:11:50.995879 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 15 12:11:50.995879 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 12:11:50.995879 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 15 12:11:50.995879 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 416/833] 10.222.0.15 () {56 vars in 1111 bytes} [Thu Sep 15 12:11:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:11:51.000315 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206a34d3-62d1-401d-af13-f15e5d3be727 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:51.026650 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c4b8a5-e98e-4e11-a7da-33481a485dab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:51.027693 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c4b8a5-e98e-4e11-a7da-33481a485dab None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:51.028185 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c4b8a5-e98e-4e11-a7da-33481a485dab None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:51.028651 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c4b8a5-e98e-4e11-a7da-33481a485dab None admin] PATH_INFO: `/v3/roles` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:51.029525 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c4b8a5-e98e-4e11-a7da-33481a485dab None admin] RBAC: Adding query filter params (name=admin) {{(pid=81449) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:11:51.030019 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c4b8a5-e98e-4e11-a7da-33481a485dab None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:11:51.031561 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c4b8a5-e98e-4e11-a7da-33481a485dab None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:11:51.041614 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 418/834] 10.222.0.15 () {56 vars in 1099 bytes} [Thu Sep 15 12:11:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:11:51.046838 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c4b8a5-e98e-4e11-a7da-33481a485dab None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:51.116056 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206a34d3-62d1-401d-af13-f15e5d3be727 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:51.152119 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6bcc72f-b782-4fea-b44b-5b742788a606 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:51.152774 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bcc72f-b782-4fea-b44b-5b742788a606 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:51.153061 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bcc72f-b782-4fea-b44b-5b742788a606 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:51.153332 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bcc72f-b782-4fea-b44b-5b742788a606 None admin] PATH_INFO: `/v3/projects` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:51.154041 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6bcc72f-b782-4fea-b44b-5b742788a606 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:11:51.154343 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6bcc72f-b782-4fea-b44b-5b742788a606 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:11:51.155803 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6bcc72f-b782-4fea-b44b-5b742788a606 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:11:51.174036 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 417/835] 10.222.0.15 () {56 vars in 1148 bytes} [Thu Sep 15 12:11:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:11:51.175130 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6bcc72f-b782-4fea-b44b-5b742788a606 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:51.178750 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c4b8a5-e98e-4e11-a7da-33481a485dab None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:51.216496 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf8cd5f-6392-4ad4-a4ef-c2683cd9d4a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:51.217541 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf8cd5f-6392-4ad4-a4ef-c2683cd9d4a0 None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:51.218018 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf8cd5f-6392-4ad4-a4ef-c2683cd9d4a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:51.218546 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf8cd5f-6392-4ad4-a4ef-c2683cd9d4a0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:51.227675 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:11:51.231691 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf8cd5f-6392-4ad4-a4ef-c2683cd9d4a0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:11:51.233443 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf8cd5f-6392-4ad4-a4ef-c2683cd9d4a0 None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:11:51.243699 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:11:51.253466 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application [None req-abf8cd5f-6392-4ad4-a4ef-c2683cd9d4a0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 12:11:51.253466 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:11:51.253466 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:11:51.253466 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:11:51.253466 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:11:51.253466 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:11:51.253466 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:11:51.253466 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:11:51.253466 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:11:51.253466 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:11:51.253466 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:11:51.253466 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:11:51.253466 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 12:11:51.253466 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 12:11:51.253466 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 12:11:51.253466 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 12:11:51.255034 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:11:51.255034 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:11:51.255034 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:11:51.255034 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:11:51.255034 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:11:51.255034 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:11:51.255034 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:11:51.255034 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application with Lock( Sep 15 12:11:51.255034 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:11:51.255034 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:11:51.255034 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:11:51.255034 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:11:51.255034 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:11:51.255034 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:11:51.255034 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:11:51.255034 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:11:51.255034 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 12:11:51.255929 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 12:11:51.255929 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 12:11:51.255929 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application Sep 15 12:11:51.258236 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 419/836] 10.222.0.15 () {56 vars in 1108 bytes} [Thu Sep 15 12:11:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:11:51.263663 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf8cd5f-6392-4ad4-a4ef-c2683cd9d4a0 None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:51.285558 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6bcc72f-b782-4fea-b44b-5b742788a606 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:51.333626 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4388000-9d09-42e1-a871-8721ed0cef00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:51.334840 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4388000-9d09-42e1-a871-8721ed0cef00 None admin] REQUEST_METHOD: `PUT` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:51.335297 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4388000-9d09-42e1-a871-8721ed0cef00 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:51.336007 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4388000-9d09-42e1-a871-8721ed0cef00 None admin] PATH_INFO: `/v3/projects/540a32ef88764e8c8d9201ae22f6ddaf/users/527f4dd6031546c785bc6360dacb60af/roles/e1cecc29e028497f9cdaa04268c6a6ec` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:51.354769 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4388000-9d09-42e1-a871-8721ed0cef00 None admin] RBAC: Authorizing `identity:create_grant(project_id=540a32ef88764e8c8d9201ae22f6ddaf, user_id=527f4dd6031546c785bc6360dacb60af, role_id=e1cecc29e028497f9cdaa04268c6a6ec)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:11:51.356295 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4388000-9d09-42e1-a871-8721ed0cef00 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:11:51.374280 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf8cd5f-6392-4ad4-a4ef-c2683cd9d4a0 None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:51.376847 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 418/837] 10.222.0.15 () {58 vars in 1439 bytes} [Thu Sep 15 12:11:51 2022] PUT /identity/v3/projects/540a32ef88764e8c8d9201ae22f6ddaf/users/527f4dd6031546c785bc6360dacb60af/roles/e1cecc29e028497f9cdaa04268c6a6ec => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:11:51.423974 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f554d21c-671c-4b87-bd58-750c2a27b5ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:51.425075 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f554d21c-671c-4b87-bd58-750c2a27b5ce None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:51.425674 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f554d21c-671c-4b87-bd58-750c2a27b5ce None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:51.426214 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f554d21c-671c-4b87-bd58-750c2a27b5ce None admin] PATH_INFO: `/v3/domains` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:51.428923 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f554d21c-671c-4b87-bd58-750c2a27b5ce None admin] RBAC: Adding query filter params (name=Default) {{(pid=81449) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:11:51.430344 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f554d21c-671c-4b87-bd58-750c2a27b5ce None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:11:51.432110 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f554d21c-671c-4b87-bd58-750c2a27b5ce None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:11:51.453802 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 420/838] 10.222.0.15 () {56 vars in 1109 bytes} [Thu Sep 15 12:11:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 12:11:51.461104 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4388000-9d09-42e1-a871-8721ed0cef00 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:51.539272 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784c247c-4ab3-4414-a5b7-5c29bf96445f None None] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:51.539591 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784c247c-4ab3-4414-a5b7-5c29bf96445f None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:51.539774 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784c247c-4ab3-4414-a5b7-5c29bf96445f None None] PATH_INFO: `/` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:51.541439 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 421/839] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 12:11:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:11:51.550730 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ddab71-66fb-4fdd-b9b8-bdf33b30b295 None None] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:51.551211 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ddab71-66fb-4fdd-b9b8-bdf33b30b295 None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:51.551801 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ddab71-66fb-4fdd-b9b8-bdf33b30b295 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:51.571936 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.auth.core [None req-70ddab71-66fb-4fdd-b9b8-bdf33b30b295 None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81449) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:11:51.573479 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-70ddab71-66fb-4fdd-b9b8-bdf33b30b295 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:11:51.580024 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4388000-9d09-42e1-a871-8721ed0cef00 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:51.582791 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:11:51.622548 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b489fb-a23e-4846-ab53-496668d1f0e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:51.623574 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b489fb-a23e-4846-ab53-496668d1f0e0 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:51.624053 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b489fb-a23e-4846-ab53-496668d1f0e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:51.624519 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b489fb-a23e-4846-ab53-496668d1f0e0 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:51.637415 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b489fb-a23e-4846-ab53-496668d1f0e0 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:11:51.638997 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b489fb-a23e-4846-ab53-496668d1f0e0 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:11:51.649705 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:11:51.653178 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application [None req-58b489fb-a23e-4846-ab53-496668d1f0e0 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Sep 15 12:11:51.653178 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:11:51.653178 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:11:51.653178 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:11:51.653178 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:11:51.653178 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:11:51.653178 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:11:51.653178 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:11:51.653178 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:11:51.653178 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:11:51.653178 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:11:51.653178 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:11:51.653178 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 15 12:11:51.653178 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 15 12:11:51.653178 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 15 12:11:51.653178 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 15 12:11:51.654103 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:11:51.654103 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:11:51.654103 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 15 12:11:51.654103 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 12:11:51.654103 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 15 12:11:51.654103 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 12:11:51.654103 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:11:51.654103 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:11:51.654103 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:11:51.654103 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:11:51.654103 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:11:51.654103 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application with Lock( Sep 15 12:11:51.654103 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:11:51.654103 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:11:51.654103 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:11:51.654103 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:11:51.654103 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:11:51.654103 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:11:51.655171 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:11:51.655171 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:11:51.655171 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 15 12:11:51.655171 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 15 12:11:51.655171 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 15 12:11:51.655171 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 15 12:11:51.655171 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 15 12:11:51.655171 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 15 12:11:51.655171 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Sep 15 12:11:51.655171 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 15 12:11:51.655171 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 419/840] 10.222.0.15 () {56 vars in 1096 bytes} [Thu Sep 15 12:11:51 2022] GET /identity/v3/users/swift => generated 83 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:11:51.661114 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b489fb-a23e-4846-ab53-496668d1f0e0 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:51.665882 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 422/841] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 12:11:51 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 15 12:11:51.766896 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b489fb-a23e-4846-ab53-496668d1f0e0 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:51.803956 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71303eb-d485-4f84-8182-79bd6ef1b19e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:51.804503 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71303eb-d485-4f84-8182-79bd6ef1b19e None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:51.804799 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71303eb-d485-4f84-8182-79bd6ef1b19e None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:51.805077 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71303eb-d485-4f84-8182-79bd6ef1b19e None admin] PATH_INFO: `/v3/users` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:51.805800 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71303eb-d485-4f84-8182-79bd6ef1b19e None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:11:51.806104 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71303eb-d485-4f84-8182-79bd6ef1b19e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:11:51.807517 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71303eb-d485-4f84-8182-79bd6ef1b19e None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:11:51.825788 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 420/842] 10.222.0.15 () {56 vars in 1135 bytes} [Thu Sep 15 12:11:51 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 12:11:51.831360 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ddab71-66fb-4fdd-b9b8-bdf33b30b295 None None] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:51.931959 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ddab71-66fb-4fdd-b9b8-bdf33b30b295 None None] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:51.966863 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71303eb-d485-4f84-8182-79bd6ef1b19e None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:51.967058 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be191b7-f001-46f9-a4c5-a7396a7b1ff4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:51.967620 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be191b7-f001-46f9-a4c5-a7396a7b1ff4 None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:51.967884 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be191b7-f001-46f9-a4c5-a7396a7b1ff4 None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:51.968292 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be191b7-f001-46f9-a4c5-a7396a7b1ff4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:51.998221 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:11:52.038713 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:11:52.058174 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be191b7-f001-46f9-a4c5-a7396a7b1ff4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:11:52.059788 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be191b7-f001-46f9-a4c5-a7396a7b1ff4 None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:11:52.097648 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71303eb-d485-4f84-8182-79bd6ef1b19e None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:52.138592 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d04d81-c7bf-49a5-9333-049fc4a6662a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:52.139585 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d04d81-c7bf-49a5-9333-049fc4a6662a None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:52.140159 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d04d81-c7bf-49a5-9333-049fc4a6662a None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:52.140651 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d04d81-c7bf-49a5-9333-049fc4a6662a None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:52.141567 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d04d81-c7bf-49a5-9333-049fc4a6662a None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:11:52.143111 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d04d81-c7bf-49a5-9333-049fc4a6662a None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:11:52.152927 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application [None req-a1d04d81-c7bf-49a5-9333-049fc4a6662a None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 15 12:11:52.152927 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:11:52.152927 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 15 12:11:52.152927 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 15 12:11:52.152927 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 15 12:11:52.152927 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 15 12:11:52.152927 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 15 12:11:52.152927 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 15 12:11:52.152927 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 15 12:11:52.152927 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 15 12:11:52.152927 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 15 12:11:52.152927 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 15 12:11:52.152927 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:11:52.152927 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:11:52.152927 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:11:52.152927 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:11:52.152927 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:11:52.153950 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:11:52.153950 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:11:52.153950 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:11:52.153950 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:11:52.153950 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:11:52.153950 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:11:52.153950 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 15 12:11:52.153950 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 15 12:11:52.153950 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 15 12:11:52.153950 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 15 12:11:52.153950 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:11:52.153950 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:11:52.153950 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:11:52.153950 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:11:52.153950 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:11:52.153950 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:11:52.154800 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:11:52.154800 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application with Lock( Sep 15 12:11:52.154800 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:11:52.154800 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:11:52.154800 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:11:52.154800 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:11:52.154800 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:11:52.154800 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:11:52.154800 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:11:52.154800 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:11:52.154800 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 15 12:11:52.154800 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 15 12:11:52.154800 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 15 12:11:52.154800 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 15 12:11:52.157684 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 421/843] 10.222.0.15 () {56 vars in 1114 bytes} [Thu Sep 15 12:11:51 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 15 12:11:52.163279 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d04d81-c7bf-49a5-9333-049fc4a6662a None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:52.179993 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 423/844] 10.222.0.15 () {58 vars in 1282 bytes} [Thu Sep 15 12:11:51 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 15 12:11:52.185213 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be191b7-f001-46f9-a4c5-a7396a7b1ff4 None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:52.258372 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:11:52.277975 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d04d81-c7bf-49a5-9333-049fc4a6662a None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:52.288337 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:11:52.302207 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be191b7-f001-46f9-a4c5-a7396a7b1ff4 None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:52.314044 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b543f28-9dd8-44f2-bbb0-873009156434 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:52.314749 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b543f28-9dd8-44f2-bbb0-873009156434 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:52.315089 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b543f28-9dd8-44f2-bbb0-873009156434 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:52.315466 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b543f28-9dd8-44f2-bbb0-873009156434 None admin] PATH_INFO: `/v3/services` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:52.316162 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b543f28-9dd8-44f2-bbb0-873009156434 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:11:52.317092 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b543f28-9dd8-44f2-bbb0-873009156434 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:11:52.318500 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b543f28-9dd8-44f2-bbb0-873009156434 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:11:52.328107 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 422/845] 10.222.0.15 () {56 vars in 1114 bytes} [Thu Sep 15 12:11:52 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 12:11:52.329135 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b543f28-9dd8-44f2-bbb0-873009156434 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:52.347735 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7431d0d4-f2b9-4da6-944f-fcc284a3e61e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:52.349017 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7431d0d4-f2b9-4da6-944f-fcc284a3e61e None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:52.349513 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7431d0d4-f2b9-4da6-944f-fcc284a3e61e None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:52.350248 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7431d0d4-f2b9-4da6-944f-fcc284a3e61e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:52.364857 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7431d0d4-f2b9-4da6-944f-fcc284a3e61e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:11:52.367151 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7431d0d4-f2b9-4da6-944f-fcc284a3e61e None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:11:52.390491 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application [None req-7431d0d4-f2b9-4da6-944f-fcc284a3e61e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 12:11:52.390491 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:11:52.390491 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:11:52.390491 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:11:52.390491 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:11:52.390491 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:11:52.390491 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:11:52.390491 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:11:52.390491 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:11:52.390491 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:11:52.390491 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:11:52.390491 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:11:52.390491 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 12:11:52.390491 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 12:11:52.390491 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 12:11:52.390491 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 12:11:52.391529 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:11:52.391529 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:11:52.391529 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:11:52.391529 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:11:52.391529 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:11:52.391529 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:11:52.391529 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:11:52.391529 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application with Lock( Sep 15 12:11:52.391529 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:11:52.391529 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:11:52.391529 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:11:52.391529 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:11:52.391529 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:11:52.391529 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:11:52.391529 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:11:52.391529 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:11:52.391529 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 12:11:52.392446 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 12:11:52.392446 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 12:11:52.392446 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application Sep 15 12:11:52.400950 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 424/846] 10.222.0.15 () {56 vars in 1108 bytes} [Thu Sep 15 12:11:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:11:52.401859 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7431d0d4-f2b9-4da6-944f-fcc284a3e61e None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:52.461764 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b543f28-9dd8-44f2-bbb0-873009156434 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:52.504353 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3573e89-829c-4a6b-ae22-fce6848882b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:52.505491 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3573e89-829c-4a6b-ae22-fce6848882b5 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:52.506054 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3573e89-829c-4a6b-ae22-fce6848882b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:52.506526 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3573e89-829c-4a6b-ae22-fce6848882b5 None admin] PATH_INFO: `/v3/services` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:52.507363 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3573e89-829c-4a6b-ae22-fce6848882b5 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:11:52.508138 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3573e89-829c-4a6b-ae22-fce6848882b5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:11:52.510250 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3573e89-829c-4a6b-ae22-fce6848882b5 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:11:52.514930 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 423/847] 10.222.0.15 () {56 vars in 1114 bytes} [Thu Sep 15 12:11:52 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:11:52.515047 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7431d0d4-f2b9-4da6-944f-fcc284a3e61e None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:52.515943 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaaa164c-8dbb-44af-b232-b5c874347a0a None None] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:52.516224 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaaa164c-8dbb-44af-b232-b5c874347a0a None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:52.516720 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaaa164c-8dbb-44af-b232-b5c874347a0a None None] PATH_INFO: `/` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:52.517794 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 424/848] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 12:11:52 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:11:52.526213 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ae8a2f-0cbd-4977-9747-c4a5016037d5 None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:52.526591 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ae8a2f-0cbd-4977-9747-c4a5016037d5 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:52.526871 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ae8a2f-0cbd-4977-9747-c4a5016037d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:52.546097 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-e6ae8a2f-0cbd-4977-9747-c4a5016037d5 None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:11:52.547334 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-e6ae8a2f-0cbd-4977-9747-c4a5016037d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:11:52.558106 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:11:52.564558 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37eed051-8b7a-4ae3-bc3b-1df7d846d083 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:52.565276 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37eed051-8b7a-4ae3-bc3b-1df7d846d083 None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:52.565640 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37eed051-8b7a-4ae3-bc3b-1df7d846d083 None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:52.565814 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37eed051-8b7a-4ae3-bc3b-1df7d846d083 None admin] PATH_INFO: `/v3/domains` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:52.566555 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37eed051-8b7a-4ae3-bc3b-1df7d846d083 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81449) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:11:52.566856 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37eed051-8b7a-4ae3-bc3b-1df7d846d083 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:11:52.568270 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37eed051-8b7a-4ae3-bc3b-1df7d846d083 None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:11:52.584814 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 425/849] 10.222.0.15 () {56 vars in 1109 bytes} [Thu Sep 15 12:11:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:11:52.588291 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37eed051-8b7a-4ae3-bc3b-1df7d846d083 None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:52.595172 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:11:52.667040 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 425/850] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 12:11:52 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 15 12:11:52.716157 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37eed051-8b7a-4ae3-bc3b-1df7d846d083 None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:52.752978 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364c79a4-0062-4091-a149-d0a74ae51f23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:52.753621 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364c79a4-0062-4091-a149-d0a74ae51f23 None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:52.754511 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364c79a4-0062-4091-a149-d0a74ae51f23 None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:52.754783 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364c79a4-0062-4091-a149-d0a74ae51f23 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:52.762144 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364c79a4-0062-4091-a149-d0a74ae51f23 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:11:52.763655 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364c79a4-0062-4091-a149-d0a74ae51f23 None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:11:52.775273 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application [None req-364c79a4-0062-4091-a149-d0a74ae51f23 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 12:11:52.775273 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:11:52.775273 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:11:52.775273 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:11:52.775273 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:11:52.775273 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:11:52.775273 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:11:52.775273 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:11:52.775273 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:11:52.775273 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:11:52.775273 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:11:52.775273 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:11:52.775273 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 15 12:11:52.775273 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 15 12:11:52.775273 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 15 12:11:52.776159 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 15 12:11:52.776159 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:11:52.776159 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:11:52.776159 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:11:52.776159 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:11:52.776159 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:11:52.776159 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:11:52.776159 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:11:52.776159 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application with Lock( Sep 15 12:11:52.776159 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:11:52.776159 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:11:52.776159 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:11:52.776159 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:11:52.776159 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:11:52.776159 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:11:52.776159 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:11:52.776159 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:11:52.777096 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 15 12:11:52.777096 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 15 12:11:52.777096 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 15 12:11:52.777096 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 15 12:11:52.777096 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 15 12:11:52.777096 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 15 12:11:52.777096 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 12:11:52.777096 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application Sep 15 12:11:52.777096 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 426/851] 10.222.0.15 () {56 vars in 1111 bytes} [Thu Sep 15 12:11:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:11:52.782546 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ae8a2f-0cbd-4977-9747-c4a5016037d5 None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:52.891674 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ae8a2f-0cbd-4977-9747-c4a5016037d5 None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:52.927560 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e4a31f-bbd8-411e-82a3-0a7fe9bf5a10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:52.928172 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e4a31f-bbd8-411e-82a3-0a7fe9bf5a10 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:52.928454 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e4a31f-bbd8-411e-82a3-0a7fe9bf5a10 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:52.928723 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e4a31f-bbd8-411e-82a3-0a7fe9bf5a10 None admin] PATH_INFO: `/v3/projects` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:52.929431 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e4a31f-bbd8-411e-82a3-0a7fe9bf5a10 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:11:52.929731 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e4a31f-bbd8-411e-82a3-0a7fe9bf5a10 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:11:52.931139 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e4a31f-bbd8-411e-82a3-0a7fe9bf5a10 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:11:52.943593 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 426/852] 10.222.0.15 () {56 vars in 1148 bytes} [Thu Sep 15 12:11:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:11:52.950032 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364c79a4-0062-4091-a149-d0a74ae51f23 None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:52.956470 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e4a31f-bbd8-411e-82a3-0a7fe9bf5a10 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:52.959250 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:11:53.030721 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:11:53.063503 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e4a31f-bbd8-411e-82a3-0a7fe9bf5a10 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:53.081368 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364c79a4-0062-4091-a149-d0a74ae51f23 None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:53.108822 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99c4b0d-bdbb-4d87-b1a8-a2a4a8a8d744 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:53.109769 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99c4b0d-bdbb-4d87-b1a8-a2a4a8a8d744 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:53.110196 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99c4b0d-bdbb-4d87-b1a8-a2a4a8a8d744 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:53.110648 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99c4b0d-bdbb-4d87-b1a8-a2a4a8a8d744 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:53.111469 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99c4b0d-bdbb-4d87-b1a8-a2a4a8a8d744 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:11:53.113159 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99c4b0d-bdbb-4d87-b1a8-a2a4a8a8d744 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:11:53.123589 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application [None req-b99c4b0d-bdbb-4d87-b1a8-a2a4a8a8d744 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 15 12:11:53.123589 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:11:53.123589 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 15 12:11:53.123589 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 15 12:11:53.123589 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 15 12:11:53.123589 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 15 12:11:53.123589 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 15 12:11:53.123589 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 15 12:11:53.123589 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 15 12:11:53.123589 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 15 12:11:53.123589 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 15 12:11:53.123589 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 15 12:11:53.123589 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:11:53.123589 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:11:53.123589 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:11:53.123589 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:11:53.124780 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:11:53.124780 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:11:53.124780 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:11:53.124780 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:11:53.124780 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:11:53.124780 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:11:53.124780 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:11:53.124780 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 15 12:11:53.124780 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 15 12:11:53.124780 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 15 12:11:53.124780 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 15 12:11:53.124780 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:11:53.124780 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:11:53.124780 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:11:53.124780 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:11:53.124780 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:11:53.124780 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:11:53.125720 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:11:53.125720 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application with Lock( Sep 15 12:11:53.125720 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:11:53.125720 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:11:53.125720 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:11:53.125720 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:11:53.125720 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:11:53.125720 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:11:53.125720 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:11:53.125720 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:11:53.125720 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 15 12:11:53.125720 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 15 12:11:53.125720 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 15 12:11:53.125720 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 15 12:11:53.126461 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 427/853] 10.222.0.15 () {56 vars in 1132 bytes} [Thu Sep 15 12:11:52 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 15 12:11:53.132060 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b5c09b-b80d-4355-8264-59cb9337c566 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:53.132060 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b5c09b-b80d-4355-8264-59cb9337c566 None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:53.132060 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b5c09b-b80d-4355-8264-59cb9337c566 None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:53.132060 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b5c09b-b80d-4355-8264-59cb9337c566 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:53.132398 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b5c09b-b80d-4355-8264-59cb9337c566 None admin] RBAC: Adding query filter params (role.id=9c769b5c82884766b2c510b1f0f1364c, scope.project.id=540a32ef88764e8c8d9201ae22f6ddaf, user.id=efa0d3dc5b7540d59fdd59bbf64a04b6) {{(pid=81449) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:11:53.132398 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b5c09b-b80d-4355-8264-59cb9337c566 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:11:53.132398 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b5c09b-b80d-4355-8264-59cb9337c566 None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:11:53.132887 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99c4b0d-bdbb-4d87-b1a8-a2a4a8a8d744 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:53.148910 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 427/854] 10.222.0.15 () {56 vars in 1374 bytes} [Thu Sep 15 12:11:52 2022] GET /identity/v3/role_assignments?user.id=efa0d3dc5b7540d59fdd59bbf64a04b6&scope.project.id=540a32ef88764e8c8d9201ae22f6ddaf&role.id=9c769b5c82884766b2c510b1f0f1364c => generated 608 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:11:53.209863 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:11:53.250275 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99c4b0d-bdbb-4d87-b1a8-a2a4a8a8d744 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:53.288690 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55eeab45-d017-4c86-bbed-561141c15826 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:53.289348 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55eeab45-d017-4c86-bbed-561141c15826 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:53.289625 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55eeab45-d017-4c86-bbed-561141c15826 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:53.289896 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55eeab45-d017-4c86-bbed-561141c15826 None admin] PATH_INFO: `/v3/services` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:53.290538 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55eeab45-d017-4c86-bbed-561141c15826 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:11:53.290843 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55eeab45-d017-4c86-bbed-561141c15826 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:11:53.292233 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55eeab45-d017-4c86-bbed-561141c15826 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:11:53.297900 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 428/855] 10.222.0.15 () {56 vars in 1126 bytes} [Thu Sep 15 12:11:53 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 12:11:53.305934 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b5c09b-b80d-4355-8264-59cb9337c566 None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:53.415256 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b5c09b-b80d-4355-8264-59cb9337c566 None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:53.441137 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:11:53.451596 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cfc5152-e2eb-43f0-be4a-2f25c9bb1b00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:53.452311 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfc5152-e2eb-43f0-be4a-2f25c9bb1b00 None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:53.452594 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfc5152-e2eb-43f0-be4a-2f25c9bb1b00 None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:53.452889 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cfc5152-e2eb-43f0-be4a-2f25c9bb1b00 None admin] PATH_INFO: `/v3/services` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:53.453533 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cfc5152-e2eb-43f0-be4a-2f25c9bb1b00 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81449) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:11:53.453831 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cfc5152-e2eb-43f0-be4a-2f25c9bb1b00 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:11:53.455242 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cfc5152-e2eb-43f0-be4a-2f25c9bb1b00 None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:11:53.465287 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 428/856] 10.222.0.15 () {56 vars in 1126 bytes} [Thu Sep 15 12:11:53 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:11:53.687497 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf71ccf9-1164-4374-ad4e-7f3bdb8a0cb8 None None] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:53.687783 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf71ccf9-1164-4374-ad4e-7f3bdb8a0cb8 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:53.688058 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf71ccf9-1164-4374-ad4e-7f3bdb8a0cb8 None None] PATH_INFO: `/` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:53.689639 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 429/857] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 12:11:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:11:53.695128 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e3158e-94ce-435b-943b-d5582bd3444f None None] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:53.695527 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e3158e-94ce-435b-943b-d5582bd3444f None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:53.695905 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e3158e-94ce-435b-943b-d5582bd3444f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:53.715765 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.auth.core [None req-98e3158e-94ce-435b-943b-d5582bd3444f None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81449) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:11:53.716982 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-98e3158e-94ce-435b-943b-d5582bd3444f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:11:53.785483 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 429/858] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 12:11:53 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 15 12:11:54.055802 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf71ccf9-1164-4374-ad4e-7f3bdb8a0cb8 None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:54.165069 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf71ccf9-1164-4374-ad4e-7f3bdb8a0cb8 None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:54.195678 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:11:54.202683 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6f49d4-63f4-404d-8e93-c942596a1f39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:54.203331 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6f49d4-63f4-404d-8e93-c942596a1f39 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:54.203610 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6f49d4-63f4-404d-8e93-c942596a1f39 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:54.203879 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6f49d4-63f4-404d-8e93-c942596a1f39 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:54.214794 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6f49d4-63f4-404d-8e93-c942596a1f39 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 12:11:54.219548 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application [None req-7e6f49d4-63f4-404d-8e93-c942596a1f39 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 15 12:11:54.219548 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:11:54.219548 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:11:54.219548 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:11:54.219548 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:11:54.219548 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:11:54.219548 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:11:54.219548 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:11:54.219548 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:11:54.219548 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:11:54.219548 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:11:54.219548 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:11:54.219548 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 12:11:54.219548 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 12:11:54.219548 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 12:11:54.219548 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 12:11:54.220459 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 12:11:54.220459 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 12:11:54.220459 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 12:11:54.220459 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 12:11:54.220459 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:11:54.220459 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:11:54.220459 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:11:54.220459 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:11:54.220459 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:11:54.220459 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:11:54.220459 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:11:54.220459 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application with Lock( Sep 15 12:11:54.220459 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:11:54.220459 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:11:54.220459 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:11:54.220459 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:11:54.221371 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:11:54.221371 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:11:54.221371 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:11:54.221371 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:11:54.221371 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 12:11:54.221371 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 12:11:54.221371 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 12:11:54.221371 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 12:11:54.221371 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 12:11:54.221371 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 12:11:54.221371 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 15 12:11:54.221371 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 15 12:11:54.221371 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 430/859] 10.222.0.15 () {56 vars in 1102 bytes} [Thu Sep 15 12:11:54 2022] GET /identity/v3/roles/service => generated 85 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:11:54.226848 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e3158e-94ce-435b-943b-d5582bd3444f None None] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:54.331231 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e3158e-94ce-435b-943b-d5582bd3444f None None] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:54.351152 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:11:54.367142 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac9f2f8-505b-4c29-8a3c-6e1646a2da19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:54.367752 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac9f2f8-505b-4c29-8a3c-6e1646a2da19 None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:54.368030 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac9f2f8-505b-4c29-8a3c-6e1646a2da19 None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:54.368299 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac9f2f8-505b-4c29-8a3c-6e1646a2da19 None admin] PATH_INFO: `/v3/roles` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:54.368991 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac9f2f8-505b-4c29-8a3c-6e1646a2da19 None admin] RBAC: Adding query filter params (name=service) {{(pid=81449) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:11:54.369288 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac9f2f8-505b-4c29-8a3c-6e1646a2da19 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:11:54.370662 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac9f2f8-505b-4c29-8a3c-6e1646a2da19 None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:11:54.379908 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 430/860] 10.222.0.15 () {56 vars in 1103 bytes} [Thu Sep 15 12:11:54 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:11:54.385838 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6f49d4-63f4-404d-8e93-c942596a1f39 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:54.494387 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6f49d4-63f4-404d-8e93-c942596a1f39 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:54.529899 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934361b3-e9b5-438b-a2ba-c10dab4c889a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:54.530540 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934361b3-e9b5-438b-a2ba-c10dab4c889a None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:54.530817 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934361b3-e9b5-438b-a2ba-c10dab4c889a None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:54.531086 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934361b3-e9b5-438b-a2ba-c10dab4c889a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:54.541644 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-934361b3-e9b5-438b-a2ba-c10dab4c889a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:11:54.543096 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-934361b3-e9b5-438b-a2ba-c10dab4c889a None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:11:54.558071 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application [None req-934361b3-e9b5-438b-a2ba-c10dab4c889a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 12:11:54.558071 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:11:54.558071 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:11:54.558071 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:11:54.558071 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:11:54.558071 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:11:54.558071 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:11:54.558071 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:11:54.558071 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:11:54.558071 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:11:54.558071 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:11:54.558071 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:11:54.558071 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 12:11:54.558071 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 12:11:54.558071 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 12:11:54.558071 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 12:11:54.559031 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:11:54.559031 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:11:54.559031 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:11:54.559031 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:11:54.559031 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:11:54.559031 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:11:54.559031 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:11:54.559031 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application with Lock( Sep 15 12:11:54.559031 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:11:54.559031 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:11:54.559031 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:11:54.559031 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:11:54.559031 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:11:54.559031 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:11:54.559031 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:11:54.559031 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:11:54.559031 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 12:11:54.559937 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 12:11:54.559937 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 12:11:54.559937 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 15 12:11:54.559937 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 431/861] 10.222.0.15 () {56 vars in 1108 bytes} [Thu Sep 15 12:11:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:11:54.565067 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac9f2f8-505b-4c29-8a3c-6e1646a2da19 None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:54.673591 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac9f2f8-505b-4c29-8a3c-6e1646a2da19 None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:54.713040 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76c5c52-a32c-413f-a221-521e58999de6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:54.713647 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76c5c52-a32c-413f-a221-521e58999de6 None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:54.713923 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76c5c52-a32c-413f-a221-521e58999de6 None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:54.714189 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76c5c52-a32c-413f-a221-521e58999de6 None admin] PATH_INFO: `/v3/domains` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:54.714822 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76c5c52-a32c-413f-a221-521e58999de6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81449) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:11:54.715120 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76c5c52-a32c-413f-a221-521e58999de6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:11:54.716478 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76c5c52-a32c-413f-a221-521e58999de6 None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:11:54.728825 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 431/862] 10.222.0.15 () {56 vars in 1109 bytes} [Thu Sep 15 12:11:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:11:54.732187 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a2f20e-0bb0-4cab-9d1e-5b2877562b2f None None] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:54.732475 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a2f20e-0bb0-4cab-9d1e-5b2877562b2f None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:54.732744 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a2f20e-0bb0-4cab-9d1e-5b2877562b2f None None] PATH_INFO: `/` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:54.734091 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 432/863] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 12:11:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:11:54.734227 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76c5c52-a32c-413f-a221-521e58999de6 None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:54.740271 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1606e2-0eee-4739-9109-57da06994c7f None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:54.740553 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1606e2-0eee-4739-9109-57da06994c7f None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:54.740856 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1606e2-0eee-4739-9109-57da06994c7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:54.763809 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-aa1606e2-0eee-4739-9109-57da06994c7f None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:11:54.765254 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-aa1606e2-0eee-4739-9109-57da06994c7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:11:54.785177 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:11:54.811596 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:11:54.842061 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76c5c52-a32c-413f-a221-521e58999de6 None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:54.854609 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 433/864] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 12:11:54 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 15 12:11:54.883074 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbaa51dd-1c2a-4b84-b3ae-415c5c58b907 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:54.883792 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbaa51dd-1c2a-4b84-b3ae-415c5c58b907 None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:54.884131 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbaa51dd-1c2a-4b84-b3ae-415c5c58b907 None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:54.884468 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbaa51dd-1c2a-4b84-b3ae-415c5c58b907 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:54.891802 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbaa51dd-1c2a-4b84-b3ae-415c5c58b907 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:11:54.893517 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbaa51dd-1c2a-4b84-b3ae-415c5c58b907 None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:11:54.906062 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application [None req-bbaa51dd-1c2a-4b84-b3ae-415c5c58b907 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 15 12:11:54.906062 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:11:54.906062 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:11:54.906062 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:11:54.906062 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:11:54.906062 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:11:54.906062 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:11:54.906062 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:11:54.906062 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:11:54.906062 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:11:54.906062 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:11:54.906062 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:11:54.906062 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 15 12:11:54.906062 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 15 12:11:54.906062 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 15 12:11:54.906062 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 15 12:11:54.906994 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:11:54.906994 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:11:54.906994 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 15 12:11:54.906994 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 12:11:54.906994 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 15 12:11:54.906994 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 12:11:54.906994 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:11:54.906994 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:11:54.906994 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:11:54.906994 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:11:54.906994 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:11:54.906994 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application with Lock( Sep 15 12:11:54.906994 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:11:54.906994 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:11:54.906994 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:11:54.906994 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:11:54.906994 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:11:54.906994 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:11:54.907951 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:11:54.907951 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:11:54.907951 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 15 12:11:54.907951 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 15 12:11:54.907951 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 15 12:11:54.907951 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 15 12:11:54.907951 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 15 12:11:54.907951 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 15 12:11:54.907951 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 15 12:11:54.907951 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application Sep 15 12:11:54.907951 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 432/865] 10.222.0.15 () {56 vars in 1117 bytes} [Thu Sep 15 12:11:54 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:11:54.912774 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1606e2-0eee-4739-9109-57da06994c7f None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:55.014114 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1606e2-0eee-4739-9109-57da06994c7f None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:55.047761 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b43ca0d-5dfc-4afe-82fc-ce15a7c90458 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:55.048371 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b43ca0d-5dfc-4afe-82fc-ce15a7c90458 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:55.048686 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b43ca0d-5dfc-4afe-82fc-ce15a7c90458 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:55.048982 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b43ca0d-5dfc-4afe-82fc-ce15a7c90458 None admin] PATH_INFO: `/v3/users` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:55.049696 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b43ca0d-5dfc-4afe-82fc-ce15a7c90458 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:11:55.049999 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b43ca0d-5dfc-4afe-82fc-ce15a7c90458 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:11:55.051939 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b43ca0d-5dfc-4afe-82fc-ce15a7c90458 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:11:55.070552 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 434/866] 10.222.0.15 () {56 vars in 1149 bytes} [Thu Sep 15 12:11:54 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 12:11:55.075704 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbaa51dd-1c2a-4b84-b3ae-415c5c58b907 None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:55.135158 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b43ca0d-5dfc-4afe-82fc-ce15a7c90458 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:55.171812 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:11:55.190447 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbaa51dd-1c2a-4b84-b3ae-415c5c58b907 None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:55.192700 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:11:55.228770 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee8b5fa-fff6-48c3-b004-2469adb0453f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:55.229892 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee8b5fa-fff6-48c3-b004-2469adb0453f None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:55.230241 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee8b5fa-fff6-48c3-b004-2469adb0453f None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:55.230578 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee8b5fa-fff6-48c3-b004-2469adb0453f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:55.259776 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b43ca0d-5dfc-4afe-82fc-ce15a7c90458 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:55.299285 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4396417-24b9-4f39-9514-8751786feee2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:55.299969 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4396417-24b9-4f39-9514-8751786feee2 None admin] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:55.300498 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4396417-24b9-4f39-9514-8751786feee2 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:55.301376 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4396417-24b9-4f39-9514-8751786feee2 None admin] PATH_INFO: `/v3/services` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:55.302367 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4396417-24b9-4f39-9514-8751786feee2 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:11:55.304026 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4396417-24b9-4f39-9514-8751786feee2 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:11:55.315292 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee8b5fa-fff6-48c3-b004-2469adb0453f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:11:55.316909 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee8b5fa-fff6-48c3-b004-2469adb0453f None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:11:55.326677 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 435/867] 10.222.0.15 () {60 vars in 1141 bytes} [Thu Sep 15 12:11:55 2022] POST /identity/v3/services => generated 251 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 12:11:55.449711 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 433/868] 10.222.0.15 () {58 vars in 1282 bytes} [Thu Sep 15 12:11:55 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 15 12:11:55.457984 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4396417-24b9-4f39-9514-8751786feee2 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:55.539328 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92a75a7-51b6-4c78-8d05-0396efb2fabb None None] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:55.539675 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92a75a7-51b6-4c78-8d05-0396efb2fabb None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:55.539878 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92a75a7-51b6-4c78-8d05-0396efb2fabb None None] PATH_INFO: `/` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:55.541487 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 434/869] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 12:11:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 15 12:11:55.549010 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7fae08-cad1-44a7-96fe-b62251c2b3ea None None] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:55.549010 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7fae08-cad1-44a7-96fe-b62251c2b3ea None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:55.549010 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7fae08-cad1-44a7-96fe-b62251c2b3ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:55.558056 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:11:55.561498 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:11:55.571897 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.auth.core [None req-1e7fae08-cad1-44a7-96fe-b62251c2b3ea None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81449) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:11:55.573108 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-1e7fae08-cad1-44a7-96fe-b62251c2b3ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:11:55.586591 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4396417-24b9-4f39-9514-8751786feee2 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:55.630503 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c247652-fa0b-463c-a831-d84f13739df6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:55.631544 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c247652-fa0b-463c-a831-d84f13739df6 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:55.632026 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c247652-fa0b-463c-a831-d84f13739df6 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:55.632474 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c247652-fa0b-463c-a831-d84f13739df6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:55.646251 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c247652-fa0b-463c-a831-d84f13739df6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:11:55.646781 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 435/870] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 12:11:55 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 15 12:11:55.648510 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c247652-fa0b-463c-a831-d84f13739df6 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:11:55.665913 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application [None req-1c247652-fa0b-463c-a831-d84f13739df6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 12:11:55.665913 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:11:55.665913 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:11:55.665913 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:11:55.665913 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:11:55.665913 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:11:55.665913 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:11:55.665913 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:11:55.665913 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:11:55.665913 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:11:55.665913 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:11:55.665913 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:11:55.665913 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 12:11:55.665913 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 12:11:55.665913 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 12:11:55.665913 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 12:11:55.666932 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:11:55.666932 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:11:55.666932 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:11:55.666932 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:11:55.666932 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:11:55.666932 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:11:55.666932 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:11:55.666932 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application with Lock( Sep 15 12:11:55.666932 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:11:55.666932 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:11:55.666932 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:11:55.666932 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:11:55.666932 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:11:55.666932 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:11:55.666932 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:11:55.666932 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:11:55.666932 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 12:11:55.667819 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 12:11:55.667819 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 12:11:55.667819 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 15 12:11:55.669725 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 436/871] 10.222.0.15 () {56 vars in 1108 bytes} [Thu Sep 15 12:11:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:11:55.680819 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7fae08-cad1-44a7-96fe-b62251c2b3ea None None] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:55.755207 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473e8bc7-270d-4801-a724-615c96b14f7b None None] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:55.755502 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473e8bc7-270d-4801-a724-615c96b14f7b None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:55.755755 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473e8bc7-270d-4801-a724-615c96b14f7b None None] PATH_INFO: `/` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:55.757491 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 437/872] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 12:11:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 15 12:11:55.762401 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa42720-5dca-4544-959f-a4240e84351a None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:55.762692 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa42720-5dca-4544-959f-a4240e84351a None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:55.762966 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa42720-5dca-4544-959f-a4240e84351a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:55.782053 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-3aa42720-5dca-4544-959f-a4240e84351a None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:11:55.783499 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-3aa42720-5dca-4544-959f-a4240e84351a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:11:55.784396 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7fae08-cad1-44a7-96fe-b62251c2b3ea None None] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:55.821320 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:11:55.828340 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73dcb40-9284-45a6-92ce-4c74b784af75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:55.828978 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73dcb40-9284-45a6-92ce-4c74b784af75 None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:55.829258 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73dcb40-9284-45a6-92ce-4c74b784af75 None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:55.829527 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73dcb40-9284-45a6-92ce-4c74b784af75 None admin] PATH_INFO: `/v3/domains` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:55.830165 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d73dcb40-9284-45a6-92ce-4c74b784af75 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81449) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:11:55.830466 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d73dcb40-9284-45a6-92ce-4c74b784af75 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:11:55.832404 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d73dcb40-9284-45a6-92ce-4c74b784af75 None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:11:55.848257 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 436/873] 10.222.0.15 () {56 vars in 1109 bytes} [Thu Sep 15 12:11:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:11:55.853429 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73dcb40-9284-45a6-92ce-4c74b784af75 None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:55.868766 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 438/874] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 12:11:55 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 15 12:11:55.948958 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa42720-5dca-4544-959f-a4240e84351a None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:55.978279 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73dcb40-9284-45a6-92ce-4c74b784af75 None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:56.015798 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259125d6-1132-4960-87fd-5aa8fd803e67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:56.016428 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259125d6-1132-4960-87fd-5aa8fd803e67 None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:56.016703 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259125d6-1132-4960-87fd-5aa8fd803e67 None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:56.016997 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259125d6-1132-4960-87fd-5aa8fd803e67 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:56.020234 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:11:56.025458 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259125d6-1132-4960-87fd-5aa8fd803e67 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:11:56.027022 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259125d6-1132-4960-87fd-5aa8fd803e67 None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:11:56.040105 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application [None req-259125d6-1132-4960-87fd-5aa8fd803e67 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 12:11:56.040105 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:11:56.040105 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:11:56.040105 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:11:56.040105 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:11:56.040105 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:11:56.040105 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:11:56.040105 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:11:56.040105 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:11:56.040105 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:11:56.040105 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:11:56.040105 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:11:56.040105 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 15 12:11:56.040105 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 15 12:11:56.040105 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 15 12:11:56.041034 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 15 12:11:56.041034 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:11:56.041034 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:11:56.041034 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:11:56.041034 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:11:56.041034 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:11:56.041034 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:11:56.041034 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:11:56.041034 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application with Lock( Sep 15 12:11:56.041034 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:11:56.041034 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:11:56.041034 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:11:56.041034 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:11:56.041034 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:11:56.041034 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:11:56.041034 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:11:56.041034 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:11:56.042661 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 15 12:11:56.042661 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 15 12:11:56.042661 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 15 12:11:56.042661 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 15 12:11:56.042661 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 15 12:11:56.042661 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 15 12:11:56.042661 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 12:11:56.042661 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application Sep 15 12:11:56.042661 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 437/875] 10.222.0.15 () {56 vars in 1111 bytes} [Thu Sep 15 12:11:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 15 12:11:56.046768 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259125d6-1132-4960-87fd-5aa8fd803e67 None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:56.061158 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa42720-5dca-4544-959f-a4240e84351a None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:56.099445 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d7ffad-aeab-4b53-aa15-f1d2c05bab2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:56.100195 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d7ffad-aeab-4b53-aa15-f1d2c05bab2f None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:56.100577 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d7ffad-aeab-4b53-aa15-f1d2c05bab2f None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:56.100969 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d7ffad-aeab-4b53-aa15-f1d2c05bab2f None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:56.101296 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:11:56.102150 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d7ffad-aeab-4b53-aa15-f1d2c05bab2f None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:11:56.103625 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d7ffad-aeab-4b53-aa15-f1d2c05bab2f None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:11:56.114482 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application [None req-78d7ffad-aeab-4b53-aa15-f1d2c05bab2f None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 15 12:11:56.114482 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:11:56.114482 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 15 12:11:56.114482 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 15 12:11:56.114482 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 15 12:11:56.114482 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 15 12:11:56.114482 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 15 12:11:56.114482 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 15 12:11:56.114482 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 15 12:11:56.114482 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 15 12:11:56.114482 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 15 12:11:56.114482 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 15 12:11:56.114482 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:11:56.114482 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:11:56.114482 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:11:56.114482 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:11:56.114482 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:11:56.116518 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:11:56.116518 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:11:56.116518 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:11:56.116518 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:11:56.116518 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:11:56.116518 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:11:56.116518 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 15 12:11:56.116518 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 15 12:11:56.116518 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 15 12:11:56.116518 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 15 12:11:56.116518 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:11:56.116518 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:11:56.116518 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:11:56.116518 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:11:56.116518 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:11:56.116518 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:11:56.118307 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:11:56.118307 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application with Lock( Sep 15 12:11:56.118307 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:11:56.118307 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:11:56.118307 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:11:56.118307 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:11:56.118307 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:11:56.118307 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:11:56.118307 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:11:56.118307 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:11:56.118307 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 15 12:11:56.118307 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 15 12:11:56.118307 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 15 12:11:56.118307 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 15 12:11:56.129094 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 439/876] 10.222.0.15 () {56 vars in 1126 bytes} [Thu Sep 15 12:11:55 2022] GET /identity/v3/services/object-store => generated 93 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:11:56.129094 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d7ffad-aeab-4b53-aa15-f1d2c05bab2f None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:56.151798 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259125d6-1132-4960-87fd-5aa8fd803e67 None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:56.185298 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7025d0b1-943d-4283-ae63-e37fd20dff7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:56.185902 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7025d0b1-943d-4283-ae63-e37fd20dff7e None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:56.186180 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7025d0b1-943d-4283-ae63-e37fd20dff7e None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:56.186446 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7025d0b1-943d-4283-ae63-e37fd20dff7e None admin] PATH_INFO: `/v3/projects` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:56.187125 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7025d0b1-943d-4283-ae63-e37fd20dff7e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81449) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:11:56.187425 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7025d0b1-943d-4283-ae63-e37fd20dff7e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:11:56.188893 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7025d0b1-943d-4283-ae63-e37fd20dff7e None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:11:56.201272 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 438/877] 10.222.0.15 () {56 vars in 1148 bytes} [Thu Sep 15 12:11:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:11:56.202047 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7025d0b1-943d-4283-ae63-e37fd20dff7e None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:56.243802 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d7ffad-aeab-4b53-aa15-f1d2c05bab2f None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:56.273815 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:11:56.279514 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a5636c-3041-4f76-9689-c17cb6bce7a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:56.280123 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a5636c-3041-4f76-9689-c17cb6bce7a9 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:56.280397 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a5636c-3041-4f76-9689-c17cb6bce7a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:56.280664 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a5636c-3041-4f76-9689-c17cb6bce7a9 None admin] PATH_INFO: `/v3/services` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:56.281312 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a5636c-3041-4f76-9689-c17cb6bce7a9 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:11:56.281608 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a5636c-3041-4f76-9689-c17cb6bce7a9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:11:56.282946 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a5636c-3041-4f76-9689-c17cb6bce7a9 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:11:56.288215 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 440/878] 10.222.0.15 () {56 vars in 1122 bytes} [Thu Sep 15 12:11:56 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:11:56.289763 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a5636c-3041-4f76-9689-c17cb6bce7a9 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:56.316476 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7025d0b1-943d-4283-ae63-e37fd20dff7e None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:56.333240 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:11:56.357054 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751f6444-7c9b-4d09-823f-ca300d91539c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:56.357585 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751f6444-7c9b-4d09-823f-ca300d91539c None admin] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:56.357863 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751f6444-7c9b-4d09-823f-ca300d91539c None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:56.358126 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751f6444-7c9b-4d09-823f-ca300d91539c None admin] PATH_INFO: `/v3/services` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:56.358763 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751f6444-7c9b-4d09-823f-ca300d91539c None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:11:56.360113 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751f6444-7c9b-4d09-823f-ca300d91539c None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:11:56.376570 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 439/879] 10.222.0.15 () {60 vars in 1141 bytes} [Thu Sep 15 12:11:56 2022] POST /identity/v3/services => generated 269 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 12:11:56.377615 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751f6444-7c9b-4d09-823f-ca300d91539c None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:56.416999 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a5636c-3041-4f76-9689-c17cb6bce7a9 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:56.475660 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f012b6-47d0-45f4-8611-ad619c348b3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:56.476392 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f012b6-47d0-45f4-8611-ad619c348b3d None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:56.476785 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f012b6-47d0-45f4-8611-ad619c348b3d None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:56.477083 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f012b6-47d0-45f4-8611-ad619c348b3d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:56.477921 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f012b6-47d0-45f4-8611-ad619c348b3d None admin] RBAC: Adding query filter params (role.id=e1cecc29e028497f9cdaa04268c6a6ec, scope.project.id=540a32ef88764e8c8d9201ae22f6ddaf, user.id=527f4dd6031546c785bc6360dacb60af) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:11:56.478298 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f012b6-47d0-45f4-8611-ad619c348b3d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:11:56.480179 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f012b6-47d0-45f4-8611-ad619c348b3d None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:11:56.493283 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 441/880] 10.222.0.15 () {56 vars in 1374 bytes} [Thu Sep 15 12:11:56 2022] GET /identity/v3/role_assignments?user.id=527f4dd6031546c785bc6360dacb60af&scope.project.id=540a32ef88764e8c8d9201ae22f6ddaf&role.id=e1cecc29e028497f9cdaa04268c6a6ec => generated 608 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:11:56.526716 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751f6444-7c9b-4d09-823f-ca300d91539c None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:56.569380 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42c1fc8-e5ef-47fc-8e7a-5401a2db87cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:56.570039 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42c1fc8-e5ef-47fc-8e7a-5401a2db87cd None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:56.570314 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42c1fc8-e5ef-47fc-8e7a-5401a2db87cd None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:56.570579 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42c1fc8-e5ef-47fc-8e7a-5401a2db87cd None admin] PATH_INFO: `/v3/services` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:56.571226 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c42c1fc8-e5ef-47fc-8e7a-5401a2db87cd None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81449) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:11:56.571591 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c42c1fc8-e5ef-47fc-8e7a-5401a2db87cd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:11:56.573033 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c42c1fc8-e5ef-47fc-8e7a-5401a2db87cd None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:11:56.578913 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 440/881] 10.222.0.15 () {56 vars in 1122 bytes} [Thu Sep 15 12:11:56 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:11:57.610614 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1f65a0-5585-4148-9649-2c8f822c18f5 None None] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:57.611132 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1f65a0-5585-4148-9649-2c8f822c18f5 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:57.611132 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1f65a0-5585-4148-9649-2c8f822c18f5 None None] PATH_INFO: `/` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:57.612728 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 442/882] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 12:11:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:11:57.618116 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5603f125-1fb9-41c3-8240-03519c98f107 None None] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:57.618496 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5603f125-1fb9-41c3-8240-03519c98f107 None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:57.618646 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5603f125-1fb9-41c3-8240-03519c98f107 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:57.637456 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.auth.core [None req-5603f125-1fb9-41c3-8240-03519c98f107 None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81449) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:11:57.638621 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-5603f125-1fb9-41c3-8240-03519c98f107 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:11:57.668061 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:11:57.713570 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 441/883] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 12:11:57 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 15 12:11:57.992536 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1f65a0-5585-4148-9649-2c8f822c18f5 None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:58.048980 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:11:58.097278 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1f65a0-5585-4148-9649-2c8f822c18f5 None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:58.134873 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9deaa4-6433-429f-819e-0c80253a5717 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:58.135522 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9deaa4-6433-429f-819e-0c80253a5717 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:58.135803 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9deaa4-6433-429f-819e-0c80253a5717 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:58.136075 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9deaa4-6433-429f-819e-0c80253a5717 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:58.136661 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9deaa4-6433-429f-819e-0c80253a5717 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:11:58.138053 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af9deaa4-6433-429f-819e-0c80253a5717 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:11:58.146999 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application [None req-af9deaa4-6433-429f-819e-0c80253a5717 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 15 12:11:58.146999 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:11:58.146999 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 15 12:11:58.146999 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 15 12:11:58.146999 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 15 12:11:58.146999 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 15 12:11:58.146999 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 15 12:11:58.146999 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 15 12:11:58.146999 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 15 12:11:58.146999 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 15 12:11:58.146999 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 15 12:11:58.146999 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 15 12:11:58.146999 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:11:58.146999 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:11:58.146999 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:11:58.146999 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:11:58.146999 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:11:58.148041 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:11:58.148041 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:11:58.148041 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:11:58.148041 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:11:58.148041 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:11:58.148041 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:11:58.148041 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 15 12:11:58.148041 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 15 12:11:58.148041 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 15 12:11:58.148041 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 15 12:11:58.148041 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:11:58.148041 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:11:58.148041 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:11:58.148041 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:11:58.148041 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:11:58.148041 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:11:58.149001 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:11:58.149001 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application with Lock( Sep 15 12:11:58.149001 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:11:58.149001 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:11:58.149001 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:11:58.149001 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:11:58.149001 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:11:58.149001 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:11:58.149001 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:11:58.149001 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:11:58.149001 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 15 12:11:58.149001 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 15 12:11:58.149001 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 15 12:11:58.149001 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 15 12:11:58.149001 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 443/884] 10.222.0.15 () {56 vars in 1114 bytes} [Thu Sep 15 12:11:57 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:11:58.154273 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5603f125-1fb9-41c3-8240-03519c98f107 None None] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:58.259725 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5603f125-1fb9-41c3-8240-03519c98f107 None None] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:58.296094 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f986754b-e09a-41cd-a255-29bcaeb3c9a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:58.296709 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f986754b-e09a-41cd-a255-29bcaeb3c9a0 None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:58.297012 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f986754b-e09a-41cd-a255-29bcaeb3c9a0 None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:58.297287 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f986754b-e09a-41cd-a255-29bcaeb3c9a0 None admin] PATH_INFO: `/v3/services` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:58.297910 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f986754b-e09a-41cd-a255-29bcaeb3c9a0 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81449) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:11:58.298209 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f986754b-e09a-41cd-a255-29bcaeb3c9a0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:11:58.299582 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f986754b-e09a-41cd-a255-29bcaeb3c9a0 None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:11:58.304787 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 442/885] 10.222.0.15 () {56 vars in 1114 bytes} [Thu Sep 15 12:11:58 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:11:58.310364 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9deaa4-6433-429f-819e-0c80253a5717 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:58.412127 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af9deaa4-6433-429f-819e-0c80253a5717 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:58.445295 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:11:58.458299 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30bef1e6-c900-475f-a718-097d972c4895 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:58.459278 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bef1e6-c900-475f-a718-097d972c4895 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:58.459771 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bef1e6-c900-475f-a718-097d972c4895 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:58.460253 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bef1e6-c900-475f-a718-097d972c4895 None admin] PATH_INFO: `/v3/services` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:58.461128 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30bef1e6-c900-475f-a718-097d972c4895 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:11:58.461656 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30bef1e6-c900-475f-a718-097d972c4895 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:11:58.463276 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30bef1e6-c900-475f-a718-097d972c4895 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:11:58.469031 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 444/886] 10.222.0.15 () {56 vars in 1114 bytes} [Thu Sep 15 12:11:58 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:11:58.476799 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f986754b-e09a-41cd-a255-29bcaeb3c9a0 None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:58.506177 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:11:58.584047 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f986754b-e09a-41cd-a255-29bcaeb3c9a0 None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:58.619416 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96b5195-006f-4a88-ba61-3a0ec900898f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:58.620052 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96b5195-006f-4a88-ba61-3a0ec900898f None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:58.620367 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96b5195-006f-4a88-ba61-3a0ec900898f None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:58.620547 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96b5195-006f-4a88-ba61-3a0ec900898f None admin] PATH_INFO: `/v3/endpoints` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:58.621278 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a96b5195-006f-4a88-ba61-3a0ec900898f None admin] RBAC: Adding query filter params (interface=public, service_id=e0bf02ff3ad141b9b6f66b385f8360ce, region_id=RegionOne) {{(pid=81449) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:11:58.621583 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a96b5195-006f-4a88-ba61-3a0ec900898f None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:11:58.622935 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a96b5195-006f-4a88-ba61-3a0ec900898f None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:11:58.628933 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 443/887] 10.222.0.15 () {56 vars in 1251 bytes} [Thu Sep 15 12:11:58 2022] GET /identity/v3/endpoints?service_id=e0bf02ff3ad141b9b6f66b385f8360ce&interface=public®ion_id=RegionOne => generated 194 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:11:58.634472 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30bef1e6-c900-475f-a718-097d972c4895 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:58.669326 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e09fcb1-9424-44dc-a1e5-6e8363c5e855 None None] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:58.669659 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e09fcb1-9424-44dc-a1e5-6e8363c5e855 None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:58.669879 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e09fcb1-9424-44dc-a1e5-6e8363c5e855 None None] PATH_INFO: `/` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:58.671445 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 444/888] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 12:11:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 15 12:11:58.678519 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6492d5c0-879a-417b-b8cd-6c9f74a0394a None None] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:58.678940 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6492d5c0-879a-417b-b8cd-6c9f74a0394a None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:58.679317 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6492d5c0-879a-417b-b8cd-6c9f74a0394a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:58.697982 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.auth.core [None req-6492d5c0-879a-417b-b8cd-6c9f74a0394a None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81449) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:11:58.699318 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-6492d5c0-879a-417b-b8cd-6c9f74a0394a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:11:58.745224 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30bef1e6-c900-475f-a718-097d972c4895 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:58.784226 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 445/889] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 12:11:58 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 15 12:11:58.789813 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05413e57-48cc-4c55-b969-bb56de7adb2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:58.790451 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05413e57-48cc-4c55-b969-bb56de7adb2f None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:58.790790 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05413e57-48cc-4c55-b969-bb56de7adb2f None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:58.791227 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05413e57-48cc-4c55-b969-bb56de7adb2f None admin] PATH_INFO: `/v3/services` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:58.791888 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05413e57-48cc-4c55-b969-bb56de7adb2f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:11:58.793344 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05413e57-48cc-4c55-b969-bb56de7adb2f None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:11:58.799888 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 445/890] 10.222.0.15 () {56 vars in 1087 bytes} [Thu Sep 15 12:11:58 2022] GET /identity/v3/services => generated 1268 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 12:11:58.867425 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb313b8-6d7e-4ccb-9573-46eb038cace1 None None] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:58.867603 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb313b8-6d7e-4ccb-9573-46eb038cace1 None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:58.867876 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb313b8-6d7e-4ccb-9573-46eb038cace1 None None] PATH_INFO: `/` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:58.869530 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 446/891] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 12:11:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:11:58.875117 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e3843b-8b08-4f73-b533-ef83927d5b8d None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:58.875436 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e3843b-8b08-4f73-b533-ef83927d5b8d None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:58.875671 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e3843b-8b08-4f73-b533-ef83927d5b8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:58.886467 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fd2dcd-3af0-4077-be64-e13e993460f9 None None] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:58.886823 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fd2dcd-3af0-4077-be64-e13e993460f9 None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:58.887026 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fd2dcd-3af0-4077-be64-e13e993460f9 None None] PATH_INFO: `/` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:58.888724 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 447/892] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 12:11:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 15 12:11:58.895641 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618d6d1d-e862-4cd5-aa21-cf1965235f51 None None] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:58.895943 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618d6d1d-e862-4cd5-aa21-cf1965235f51 None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:58.896208 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-618d6d1d-e862-4cd5-aa21-cf1965235f51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:58.899519 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-90e3843b-8b08-4f73-b533-ef83927d5b8d None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:11:58.900730 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-90e3843b-8b08-4f73-b533-ef83927d5b8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:11:58.916008 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.auth.core [None req-618d6d1d-e862-4cd5-aa21-cf1965235f51 None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81449) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:11:58.917239 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-618d6d1d-e862-4cd5-aa21-cf1965235f51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:11:58.939596 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:11:58.963890 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:11:58.995333 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 446/893] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 12:11:58 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 15 12:11:59.014625 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 448/894] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 12:11:58 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 15 12:11:59.063841 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e3843b-8b08-4f73-b533-ef83927d5b8d None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:59.161799 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e3843b-8b08-4f73-b533-ef83927d5b8d None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:59.200553 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc15a9f0-9fbb-4547-8bad-91774d162128 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:59.201576 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc15a9f0-9fbb-4547-8bad-91774d162128 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:59.202074 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc15a9f0-9fbb-4547-8bad-91774d162128 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:59.202555 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc15a9f0-9fbb-4547-8bad-91774d162128 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:59.203331 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc15a9f0-9fbb-4547-8bad-91774d162128 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:11:59.204908 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc15a9f0-9fbb-4547-8bad-91774d162128 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:11:59.214201 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application [None req-bc15a9f0-9fbb-4547-8bad-91774d162128 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 15 12:11:59.214201 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:11:59.214201 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 15 12:11:59.214201 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 15 12:11:59.214201 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 15 12:11:59.214201 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 15 12:11:59.214201 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 15 12:11:59.214201 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 15 12:11:59.214201 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 15 12:11:59.214201 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 15 12:11:59.214201 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 15 12:11:59.214201 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 15 12:11:59.214201 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:11:59.214201 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:11:59.214201 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:11:59.214201 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:11:59.215163 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:11:59.215163 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:11:59.215163 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:11:59.215163 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:11:59.215163 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:11:59.215163 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:11:59.215163 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:11:59.215163 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 15 12:11:59.215163 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 15 12:11:59.215163 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 15 12:11:59.215163 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 15 12:11:59.215163 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:11:59.215163 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:11:59.215163 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:11:59.215163 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:11:59.215163 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:11:59.215163 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:11:59.216065 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:11:59.216065 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application with Lock( Sep 15 12:11:59.216065 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:11:59.216065 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:11:59.216065 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:11:59.216065 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:11:59.216065 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:11:59.216065 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:11:59.216065 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:11:59.216065 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:11:59.216065 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 15 12:11:59.216065 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 15 12:11:59.216065 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 15 12:11:59.216065 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 15 12:11:59.218781 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 447/895] 10.222.0.15 () {56 vars in 1132 bytes} [Thu Sep 15 12:11:59 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:11:59.225010 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618d6d1d-e862-4cd5-aa21-cf1965235f51 None None] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:59.266764 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc15a9f0-9fbb-4547-8bad-91774d162128 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:59.319052 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:11:59.332205 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-618d6d1d-e862-4cd5-aa21-cf1965235f51 None None] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:59.368489 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ba76fc-fc69-43ab-aff9-a336a0b72397 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:59.369123 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ba76fc-fc69-43ab-aff9-a336a0b72397 None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:59.369399 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ba76fc-fc69-43ab-aff9-a336a0b72397 None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:59.369668 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ba76fc-fc69-43ab-aff9-a336a0b72397 None admin] PATH_INFO: `/v3/services` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:59.370301 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ba76fc-fc69-43ab-aff9-a336a0b72397 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81449) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:11:59.370589 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ba76fc-fc69-43ab-aff9-a336a0b72397 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:11:59.372517 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ba76fc-fc69-43ab-aff9-a336a0b72397 None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:11:59.383076 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 449/896] 10.222.0.15 () {56 vars in 1126 bytes} [Thu Sep 15 12:11:59 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:11:59.383076 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ba76fc-fc69-43ab-aff9-a336a0b72397 None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:59.387626 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc15a9f0-9fbb-4547-8bad-91774d162128 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:59.401488 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:11:59.423762 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7b9f05-8cdf-438e-903e-b857400e4cb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:59.424795 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7b9f05-8cdf-438e-903e-b857400e4cb9 None admin] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:59.424795 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7b9f05-8cdf-438e-903e-b857400e4cb9 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:59.425026 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7b9f05-8cdf-438e-903e-b857400e4cb9 None admin] PATH_INFO: `/v3/services` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:59.425650 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7b9f05-8cdf-438e-903e-b857400e4cb9 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:11:59.427002 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7b9f05-8cdf-438e-903e-b857400e4cb9 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:11:59.444801 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 448/897] 10.222.0.15 () {60 vars in 1141 bytes} [Thu Sep 15 12:11:59 2022] POST /identity/v3/services => generated 241 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 15 12:11:59.444801 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7b9f05-8cdf-438e-903e-b857400e4cb9 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:59.505097 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ba76fc-fc69-43ab-aff9-a336a0b72397 None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:59.552177 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4c761d-cd91-424d-983f-31962b5f0ff0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:59.553019 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4c761d-cd91-424d-983f-31962b5f0ff0 None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:59.553879 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4c761d-cd91-424d-983f-31962b5f0ff0 None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:59.553879 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4c761d-cd91-424d-983f-31962b5f0ff0 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:59.566135 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4c761d-cd91-424d-983f-31962b5f0ff0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 12:11:59.571396 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application [None req-8d4c761d-cd91-424d-983f-31962b5f0ff0 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 15 12:11:59.571396 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:11:59.571396 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:11:59.571396 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:11:59.571396 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:11:59.571396 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:11:59.571396 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:11:59.571396 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:11:59.571396 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:11:59.571396 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:11:59.571396 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:11:59.571396 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:11:59.571396 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 12:11:59.571396 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 12:11:59.571396 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 12:11:59.571396 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 12:11:59.572673 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 12:11:59.572673 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 12:11:59.572673 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 12:11:59.572673 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 12:11:59.572673 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:11:59.572673 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:11:59.572673 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:11:59.572673 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:11:59.572673 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:11:59.572673 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:11:59.572673 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:11:59.572673 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application with Lock( Sep 15 12:11:59.572673 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:11:59.572673 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:11:59.572673 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:11:59.572673 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:11:59.573628 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:11:59.573628 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:11:59.573628 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:11:59.573628 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:11:59.573628 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 12:11:59.573628 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 12:11:59.573628 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 12:11:59.573628 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 12:11:59.573628 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 12:11:59.573628 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 12:11:59.573628 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 15 12:11:59.573628 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application Sep 15 12:11:59.579203 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 450/898] 10.222.0.15 () {56 vars in 1120 bytes} [Thu Sep 15 12:11:59 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:11:59.580554 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4c761d-cd91-424d-983f-31962b5f0ff0 None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:59.588864 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7b9f05-8cdf-438e-903e-b857400e4cb9 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:59.591241 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:11:59.635136 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a55a58e-7da7-409b-8197-1244aa3eea4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:59.635779 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a55a58e-7da7-409b-8197-1244aa3eea4a None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:59.635970 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a55a58e-7da7-409b-8197-1244aa3eea4a None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:59.636413 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a55a58e-7da7-409b-8197-1244aa3eea4a None admin] PATH_INFO: `/v3/services` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:59.637446 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a55a58e-7da7-409b-8197-1244aa3eea4a None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:11:59.637826 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a55a58e-7da7-409b-8197-1244aa3eea4a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:11:59.639259 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a55a58e-7da7-409b-8197-1244aa3eea4a None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:11:59.646077 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 449/899] 10.222.0.15 () {56 vars in 1126 bytes} [Thu Sep 15 12:11:59 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:11:59.651944 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a55a58e-7da7-409b-8197-1244aa3eea4a None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:59.675496 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:11:59.704151 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4c761d-cd91-424d-983f-31962b5f0ff0 None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:59.743139 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce02c089-e9c3-44d6-9c4c-84f6fc69221b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:59.743928 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce02c089-e9c3-44d6-9c4c-84f6fc69221b None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:59.744304 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce02c089-e9c3-44d6-9c4c-84f6fc69221b None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:59.744667 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce02c089-e9c3-44d6-9c4c-84f6fc69221b None admin] PATH_INFO: `/v3/roles` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:59.745425 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce02c089-e9c3-44d6-9c4c-84f6fc69221b None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81449) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:11:59.745790 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce02c089-e9c3-44d6-9c4c-84f6fc69221b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:11:59.747231 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce02c089-e9c3-44d6-9c4c-84f6fc69221b None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:11:59.757688 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 451/900] 10.222.0.15 () {56 vars in 1115 bytes} [Thu Sep 15 12:11:59 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:11:59.770622 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce02c089-e9c3-44d6-9c4c-84f6fc69221b None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:59.783426 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a55a58e-7da7-409b-8197-1244aa3eea4a None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:59.821906 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeef92ce-5f1a-491e-940d-d2d3b157e3e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:59.822523 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeef92ce-5f1a-491e-940d-d2d3b157e3e5 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:59.822801 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeef92ce-5f1a-491e-940d-d2d3b157e3e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:59.823072 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeef92ce-5f1a-491e-940d-d2d3b157e3e5 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:59.823765 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeef92ce-5f1a-491e-940d-d2d3b157e3e5 None admin] RBAC: Adding query filter params (interface=public, service_id=9c08f0a558eb403fbd07e8967ebc0a36, region_id=RegionOne) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:11:59.824071 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeef92ce-5f1a-491e-940d-d2d3b157e3e5 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:11:59.825501 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeef92ce-5f1a-491e-940d-d2d3b157e3e5 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:11:59.835704 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 450/901] 10.222.0.15 () {56 vars in 1251 bytes} [Thu Sep 15 12:11:59 2022] GET /identity/v3/endpoints?service_id=9c08f0a558eb403fbd07e8967ebc0a36&interface=public®ion_id=RegionOne => generated 194 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 12:11:59.841070 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeef92ce-5f1a-491e-940d-d2d3b157e3e5 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:59.872969 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:11:59.896730 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce02c089-e9c3-44d6-9c4c-84f6fc69221b None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:59.935461 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f1f692-d2fa-45bd-a318-3758f3aa1ca7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:59.936450 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f1f692-d2fa-45bd-a318-3758f3aa1ca7 None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:59.937022 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f1f692-d2fa-45bd-a318-3758f3aa1ca7 None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:59.937443 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f1f692-d2fa-45bd-a318-3758f3aa1ca7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:11:59.940525 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:11:59.946006 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeef92ce-5f1a-491e-940d-d2d3b157e3e5 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:11:59.951512 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f1f692-d2fa-45bd-a318-3758f3aa1ca7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:11:59.953164 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f1f692-d2fa-45bd-a318-3758f3aa1ca7 None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:11:59.972956 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application [None req-59f1f692-d2fa-45bd-a318-3758f3aa1ca7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 12:11:59.972956 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:11:59.972956 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:11:59.972956 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:11:59.972956 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:11:59.972956 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:11:59.972956 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:11:59.972956 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:11:59.972956 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:11:59.972956 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:11:59.972956 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:11:59.972956 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:11:59.972956 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 12:11:59.972956 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 12:11:59.972956 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 12:11:59.972956 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 12:11:59.974185 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:11:59.974185 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:11:59.974185 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:11:59.974185 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:11:59.974185 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:11:59.974185 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:11:59.974185 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:11:59.974185 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application with Lock( Sep 15 12:11:59.974185 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:11:59.974185 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:11:59.974185 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:11:59.974185 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:11:59.974185 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:11:59.974185 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:11:59.974185 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:11:59.974185 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:11:59.974185 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 12:11:59.975128 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 12:11:59.975128 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 12:11:59.975128 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application Sep 15 12:11:59.978952 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 452/902] 10.222.0.15 () {56 vars in 1108 bytes} [Thu Sep 15 12:11:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:11:59.985636 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f1f692-d2fa-45bd-a318-3758f3aa1ca7 None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:11:59.997210 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66d4c49-f11c-429f-87b6-2b5ab1c92718 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:11:59.998537 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66d4c49-f11c-429f-87b6-2b5ab1c92718 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:11:59.999016 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66d4c49-f11c-429f-87b6-2b5ab1c92718 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:11:59.999439 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66d4c49-f11c-429f-87b6-2b5ab1c92718 None admin] PATH_INFO: `/v3/services` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:12:00.000646 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66d4c49-f11c-429f-87b6-2b5ab1c92718 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:12:00.002943 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66d4c49-f11c-429f-87b6-2b5ab1c92718 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:12:00.010935 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 451/903] 10.222.0.15 () {56 vars in 1087 bytes} [Thu Sep 15 12:11:59 2022] GET /identity/v3/services => generated 1497 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 15 12:12:00.106369 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f1f692-d2fa-45bd-a318-3758f3aa1ca7 None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:12:00.146218 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e641c8-3f80-4d9d-9c77-6effac5919a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:12:00.147093 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e641c8-3f80-4d9d-9c77-6effac5919a6 None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:12:00.147531 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e641c8-3f80-4d9d-9c77-6effac5919a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:12:00.147982 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e641c8-3f80-4d9d-9c77-6effac5919a6 None admin] PATH_INFO: `/v3/domains` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:12:00.148828 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e641c8-3f80-4d9d-9c77-6effac5919a6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81449) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:12:00.149341 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e641c8-3f80-4d9d-9c77-6effac5919a6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:12:00.150952 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e641c8-3f80-4d9d-9c77-6effac5919a6 None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:12:00.165864 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 453/904] 10.222.0.15 () {56 vars in 1109 bytes} [Thu Sep 15 12:11:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:12:00.174109 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66d4c49-f11c-429f-87b6-2b5ab1c92718 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:12:00.275873 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:12:00.282708 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66d4c49-f11c-429f-87b6-2b5ab1c92718 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:12:00.319742 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93737d8-d977-48cd-8e3f-83c56eaf67f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:12:00.320379 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93737d8-d977-48cd-8e3f-83c56eaf67f9 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:12:00.320656 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93737d8-d977-48cd-8e3f-83c56eaf67f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:12:00.320959 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93737d8-d977-48cd-8e3f-83c56eaf67f9 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:12:00.328216 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93737d8-d977-48cd-8e3f-83c56eaf67f9 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:12:00.329680 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93737d8-d977-48cd-8e3f-83c56eaf67f9 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:12:00.341457 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application [None req-c93737d8-d977-48cd-8e3f-83c56eaf67f9 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 15 12:12:00.341457 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:12:00.341457 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:12:00.341457 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:12:00.341457 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:12:00.341457 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:12:00.341457 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:12:00.341457 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:12:00.341457 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:12:00.341457 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:12:00.341457 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:12:00.341457 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:12:00.341457 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 15 12:12:00.341457 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 15 12:12:00.341457 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 15 12:12:00.341457 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 15 12:12:00.342422 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:12:00.342422 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:12:00.342422 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 15 12:12:00.342422 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 12:12:00.342422 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 15 12:12:00.342422 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 12:12:00.342422 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:12:00.342422 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:12:00.342422 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:12:00.342422 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:12:00.342422 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:12:00.342422 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application with Lock( Sep 15 12:12:00.342422 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:12:00.342422 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:12:00.342422 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:12:00.342422 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:12:00.342422 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:12:00.342422 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:12:00.343368 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:12:00.343368 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:12:00.343368 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 15 12:12:00.343368 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 15 12:12:00.343368 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 15 12:12:00.343368 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 15 12:12:00.343368 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 15 12:12:00.343368 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 15 12:12:00.343368 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 15 12:12:00.343368 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 15 12:12:00.343368 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 452/905] 10.222.0.15 () {56 vars in 1117 bytes} [Thu Sep 15 12:12:00 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:12:00.348558 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e641c8-3f80-4d9d-9c77-6effac5919a6 None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:12:00.400241 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:12:00.459441 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e641c8-3f80-4d9d-9c77-6effac5919a6 None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:12:00.498376 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170d11d6-dcc8-44d1-b28c-00dfb6cc5578 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:12:00.499051 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170d11d6-dcc8-44d1-b28c-00dfb6cc5578 None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:12:00.499335 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170d11d6-dcc8-44d1-b28c-00dfb6cc5578 None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:12:00.499605 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170d11d6-dcc8-44d1-b28c-00dfb6cc5578 None admin] PATH_INFO: `/v3/users` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:12:00.500332 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170d11d6-dcc8-44d1-b28c-00dfb6cc5578 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81449) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:12:00.500635 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170d11d6-dcc8-44d1-b28c-00dfb6cc5578 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:12:00.502091 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170d11d6-dcc8-44d1-b28c-00dfb6cc5578 None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:12:00.521662 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 454/906] 10.222.0.15 () {56 vars in 1149 bytes} [Thu Sep 15 12:12:00 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:12:00.527359 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93737d8-d977-48cd-8e3f-83c56eaf67f9 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:12:00.631790 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93737d8-d977-48cd-8e3f-83c56eaf67f9 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:12:00.666137 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663a16e4-b54a-4b6e-b654-850bdf4e7229 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:12:00.666751 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663a16e4-b54a-4b6e-b654-850bdf4e7229 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:12:00.667015 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663a16e4-b54a-4b6e-b654-850bdf4e7229 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:12:00.667288 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663a16e4-b54a-4b6e-b654-850bdf4e7229 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:12:00.692792 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:12:00.741408 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663a16e4-b54a-4b6e-b654-850bdf4e7229 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:12:00.741408 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663a16e4-b54a-4b6e-b654-850bdf4e7229 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:12:00.846398 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 453/907] 10.222.0.15 () {58 vars in 1282 bytes} [Thu Sep 15 12:12:00 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 15 12:12:00.852023 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170d11d6-dcc8-44d1-b28c-00dfb6cc5578 None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:12:00.961269 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170d11d6-dcc8-44d1-b28c-00dfb6cc5578 None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:12:00.970675 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:12:01.001378 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530c9ad3-4279-428b-81ec-dcbf0a925dcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:12:01.002012 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530c9ad3-4279-428b-81ec-dcbf0a925dcc None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:12:01.002288 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530c9ad3-4279-428b-81ec-dcbf0a925dcc None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:12:01.002560 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530c9ad3-4279-428b-81ec-dcbf0a925dcc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:12:01.013565 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530c9ad3-4279-428b-81ec-dcbf0a925dcc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:12:01.015007 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530c9ad3-4279-428b-81ec-dcbf0a925dcc None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:12:01.029676 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application [None req-530c9ad3-4279-428b-81ec-dcbf0a925dcc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 12:12:01.029676 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:12:01.029676 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:12:01.029676 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:12:01.029676 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:12:01.029676 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:12:01.029676 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:12:01.029676 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:12:01.029676 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:12:01.029676 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:12:01.029676 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:12:01.029676 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:12:01.029676 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 12:12:01.029676 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 12:12:01.029676 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 12:12:01.029676 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 12:12:01.030602 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:12:01.030602 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:12:01.030602 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:12:01.030602 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:12:01.030602 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:12:01.030602 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:12:01.030602 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:12:01.030602 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application with Lock( Sep 15 12:12:01.030602 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:12:01.030602 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:12:01.030602 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:12:01.030602 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:12:01.030602 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:12:01.030602 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:12:01.030602 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:12:01.030602 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:12:01.030602 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 12:12:01.031521 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 12:12:01.031521 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 12:12:01.031521 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application Sep 15 12:12:01.031521 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 455/908] 10.222.0.15 () {56 vars in 1108 bytes} [Thu Sep 15 12:12:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:12:01.036402 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663a16e4-b54a-4b6e-b654-850bdf4e7229 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:12:01.063873 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcea4cd-0a17-403b-9954-895368d96c1a None None] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:12:01.064162 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcea4cd-0a17-403b-9954-895368d96c1a None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:12:01.064442 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fcea4cd-0a17-403b-9954-895368d96c1a None None] PATH_INFO: `/` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:12:01.066670 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 456/909] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 12:12:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 15 12:12:01.073118 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e2c6e4-15fd-4561-8889-61b98c684b7d None None] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:12:01.073404 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e2c6e4-15fd-4561-8889-61b98c684b7d None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:12:01.073678 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e2c6e4-15fd-4561-8889-61b98c684b7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:12:01.093556 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.auth.core [None req-f5e2c6e4-15fd-4561-8889-61b98c684b7d None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81449) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:12:01.094841 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-f5e2c6e4-15fd-4561-8889-61b98c684b7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:12:01.096685 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:12:01.146202 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663a16e4-b54a-4b6e-b654-850bdf4e7229 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:12:01.187800 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29930927-8251-418b-ad4e-5f7f91242b89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:12:01.188416 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29930927-8251-418b-ad4e-5f7f91242b89 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:12:01.188695 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29930927-8251-418b-ad4e-5f7f91242b89 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:12:01.188899 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 457/910] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 12:12:01 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 15 12:12:01.189059 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29930927-8251-418b-ad4e-5f7f91242b89 None admin] PATH_INFO: `/v3/domains` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:12:01.189621 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29930927-8251-418b-ad4e-5f7f91242b89 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:12:01.189920 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29930927-8251-418b-ad4e-5f7f91242b89 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:12:01.191846 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29930927-8251-418b-ad4e-5f7f91242b89 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:12:01.207213 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 454/911] 10.222.0.15 () {56 vars in 1109 bytes} [Thu Sep 15 12:12:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:12:01.212376 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e2c6e4-15fd-4561-8889-61b98c684b7d None None] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:12:01.303355 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:12:01.315679 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e2c6e4-15fd-4561-8889-61b98c684b7d None None] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:12:01.351053 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0f1766-eda2-4469-ac32-9666683aed76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:12:01.351616 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0f1766-eda2-4469-ac32-9666683aed76 None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:12:01.351897 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0f1766-eda2-4469-ac32-9666683aed76 None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:12:01.352169 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0f1766-eda2-4469-ac32-9666683aed76 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:12:01.359285 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0f1766-eda2-4469-ac32-9666683aed76 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:12:01.360737 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c0f1766-eda2-4469-ac32-9666683aed76 None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:12:01.373488 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application [None req-3c0f1766-eda2-4469-ac32-9666683aed76 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 12:12:01.373488 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:12:01.373488 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:12:01.373488 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:12:01.373488 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:12:01.373488 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:12:01.373488 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:12:01.373488 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:12:01.373488 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:12:01.373488 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:12:01.373488 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:12:01.373488 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:12:01.373488 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 15 12:12:01.373488 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 15 12:12:01.373488 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 15 12:12:01.374494 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 15 12:12:01.374494 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:12:01.374494 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:12:01.374494 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:12:01.374494 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:12:01.374494 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:12:01.374494 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:12:01.374494 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:12:01.374494 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application with Lock( Sep 15 12:12:01.374494 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:12:01.374494 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:12:01.374494 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:12:01.374494 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:12:01.374494 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:12:01.374494 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:12:01.374494 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:12:01.374494 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:12:01.375396 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 15 12:12:01.375396 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 15 12:12:01.375396 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 15 12:12:01.375396 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 15 12:12:01.375396 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 15 12:12:01.375396 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 15 12:12:01.375396 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 12:12:01.375396 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application Sep 15 12:12:01.375396 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 458/912] 10.222.0.15 () {56 vars in 1111 bytes} [Thu Sep 15 12:12:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:12:01.380643 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29930927-8251-418b-ad4e-5f7f91242b89 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:12:01.464986 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0f1766-eda2-4469-ac32-9666683aed76 None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:12:01.485355 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29930927-8251-418b-ad4e-5f7f91242b89 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:12:01.500853 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:12:01.523195 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5640546b-3327-4f53-a865-33fa70a72f81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:12:01.523811 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5640546b-3327-4f53-a865-33fa70a72f81 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:12:01.524087 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5640546b-3327-4f53-a865-33fa70a72f81 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:12:01.524354 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5640546b-3327-4f53-a865-33fa70a72f81 None admin] PATH_INFO: `/v3/projects` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:12:01.525063 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5640546b-3327-4f53-a865-33fa70a72f81 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:12:01.525420 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5640546b-3327-4f53-a865-33fa70a72f81 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:12:01.526894 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5640546b-3327-4f53-a865-33fa70a72f81 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:12:01.539917 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 455/913] 10.222.0.15 () {56 vars in 1148 bytes} [Thu Sep 15 12:12:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:12:01.545992 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5640546b-3327-4f53-a865-33fa70a72f81 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:12:01.599733 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0f1766-eda2-4469-ac32-9666683aed76 None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:12:01.647024 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8669f88b-2630-47fd-a6d1-fc8d719a9833 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:12:01.648081 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8669f88b-2630-47fd-a6d1-fc8d719a9833 None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:12:01.649155 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8669f88b-2630-47fd-a6d1-fc8d719a9833 None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:12:01.649435 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8669f88b-2630-47fd-a6d1-fc8d719a9833 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:12:01.650058 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8669f88b-2630-47fd-a6d1-fc8d719a9833 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:12:01.652807 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8669f88b-2630-47fd-a6d1-fc8d719a9833 None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:12:01.663426 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application [None req-8669f88b-2630-47fd-a6d1-fc8d719a9833 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 15 12:12:01.663426 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:12:01.663426 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 15 12:12:01.663426 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 15 12:12:01.663426 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 15 12:12:01.663426 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 15 12:12:01.663426 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 15 12:12:01.663426 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 15 12:12:01.663426 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 15 12:12:01.663426 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application Sep 15 12:12:01.663426 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 15 12:12:01.663426 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application Sep 15 12:12:01.663426 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:12:01.663426 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:12:01.663426 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:12:01.663426 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:12:01.663426 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:12:01.664511 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:12:01.664511 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:12:01.664511 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:12:01.664511 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:12:01.664511 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:12:01.664511 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:12:01.664511 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 15 12:12:01.664511 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 15 12:12:01.664511 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 15 12:12:01.664511 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 15 12:12:01.664511 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:12:01.664511 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:12:01.664511 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:12:01.664511 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:12:01.664511 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:12:01.664511 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:12:01.665399 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:12:01.665399 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application with Lock( Sep 15 12:12:01.665399 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:12:01.665399 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:12:01.665399 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:12:01.665399 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:12:01.665399 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:12:01.665399 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:12:01.665399 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:12:01.665399 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:12:01.665399 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 15 12:12:01.665399 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 15 12:12:01.665399 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Sep 15 12:12:01.665399 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application Sep 15 12:12:01.665399 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 459/914] 10.222.0.15 () {56 vars in 1114 bytes} [Thu Sep 15 12:12:01 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 15 12:12:01.671160 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5640546b-3327-4f53-a865-33fa70a72f81 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:12:01.680046 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8669f88b-2630-47fd-a6d1-fc8d719a9833 None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:12:01.707401 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859f601d-edc5-4874-a605-f66f68d46cd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:12:01.708023 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859f601d-edc5-4874-a605-f66f68d46cd2 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:12:01.708491 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:12:01.708705 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859f601d-edc5-4874-a605-f66f68d46cd2 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:12:01.709075 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859f601d-edc5-4874-a605-f66f68d46cd2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:12:01.710082 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-859f601d-edc5-4874-a605-f66f68d46cd2 None admin] RBAC: Adding query filter params (role.id=c0320da65e0b4cdd8924c20a99b978af, scope.project.id=540a32ef88764e8c8d9201ae22f6ddaf, user.id=527f4dd6031546c785bc6360dacb60af) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:12:01.710392 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-859f601d-edc5-4874-a605-f66f68d46cd2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:12:01.711825 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-859f601d-edc5-4874-a605-f66f68d46cd2 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:12:01.724847 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 456/915] 10.222.0.15 () {56 vars in 1374 bytes} [Thu Sep 15 12:12:01 2022] GET /identity/v3/role_assignments?user.id=527f4dd6031546c785bc6360dacb60af&scope.project.id=540a32ef88764e8c8d9201ae22f6ddaf&role.id=c0320da65e0b4cdd8924c20a99b978af => generated 259 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:12:01.775994 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583c7cb9-d7d4-462d-b675-39d3ad31cdf7 None None] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:12:01.776302 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583c7cb9-d7d4-462d-b675-39d3ad31cdf7 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:12:01.776547 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583c7cb9-d7d4-462d-b675-39d3ad31cdf7 None None] PATH_INFO: `/` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:12:01.779525 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 457/916] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 12:12:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:12:01.786661 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a12088-95b0-42f7-9d42-19d58083c51e None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:12:01.786982 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a12088-95b0-42f7-9d42-19d58083c51e None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:12:01.787205 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a12088-95b0-42f7-9d42-19d58083c51e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:12:01.806390 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-95a12088-95b0-42f7-9d42-19d58083c51e None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:12:01.807620 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-95a12088-95b0-42f7-9d42-19d58083c51e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:12:01.820469 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8669f88b-2630-47fd-a6d1-fc8d719a9833 None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:12:01.842034 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:12:01.881557 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688df7d4-362c-4b9c-babe-147b958fbdd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:12:01.883450 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688df7d4-362c-4b9c-babe-147b958fbdd7 None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:12:01.884031 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688df7d4-362c-4b9c-babe-147b958fbdd7 None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:12:01.884485 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688df7d4-362c-4b9c-babe-147b958fbdd7 None admin] PATH_INFO: `/v3/services` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:12:01.885624 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688df7d4-362c-4b9c-babe-147b958fbdd7 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81449) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:12:01.886557 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688df7d4-362c-4b9c-babe-147b958fbdd7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:12:01.888794 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688df7d4-362c-4b9c-babe-147b958fbdd7 None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:12:01.894932 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 458/917] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 12:12:01 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 15 12:12:01.896559 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 460/918] 10.222.0.15 () {56 vars in 1114 bytes} [Thu Sep 15 12:12:01 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:12:01.901866 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a12088-95b0-42f7-9d42-19d58083c51e None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:12:02.006043 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a12088-95b0-42f7-9d42-19d58083c51e None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:12:02.041340 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5b32cb-90e0-43e5-a6c6-71ffb03cdb2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:12:02.041952 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5b32cb-90e0-43e5-a6c6-71ffb03cdb2d None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:12:02.042232 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5b32cb-90e0-43e5-a6c6-71ffb03cdb2d None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:12:02.042500 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5b32cb-90e0-43e5-a6c6-71ffb03cdb2d None admin] PATH_INFO: `/v3/services` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:12:02.043129 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5b32cb-90e0-43e5-a6c6-71ffb03cdb2d None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:12:02.043433 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5b32cb-90e0-43e5-a6c6-71ffb03cdb2d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:12:02.044814 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5b32cb-90e0-43e5-a6c6-71ffb03cdb2d None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:12:02.050205 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 459/919] 10.222.0.15 () {56 vars in 1114 bytes} [Thu Sep 15 12:12:01 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 12:12:02.055473 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688df7d4-362c-4b9c-babe-147b958fbdd7 None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:12:02.162134 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:12:02.167479 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688df7d4-362c-4b9c-babe-147b958fbdd7 None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:12:02.176100 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5b32cb-90e0-43e5-a6c6-71ffb03cdb2d None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:12:02.209141 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:12:02.212070 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deff2b1a-39ee-41a5-9135-7d3e49da6b52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:12:02.212804 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deff2b1a-39ee-41a5-9135-7d3e49da6b52 None admin] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:12:02.213022 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deff2b1a-39ee-41a5-9135-7d3e49da6b52 None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:12:02.213277 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deff2b1a-39ee-41a5-9135-7d3e49da6b52 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:12:02.214133 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deff2b1a-39ee-41a5-9135-7d3e49da6b52 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:12:02.217132 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deff2b1a-39ee-41a5-9135-7d3e49da6b52 None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:12:02.247229 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 461/920] 10.222.0.15 () {60 vars in 1144 bytes} [Thu Sep 15 12:12:02 2022] POST /identity/v3/endpoints => generated 346 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 12:12:02.296534 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5b32cb-90e0-43e5-a6c6-71ffb03cdb2d None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:12:02.341956 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78352d98-c00e-4b9e-bced-d4773ade6abe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:12:02.342866 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78352d98-c00e-4b9e-bced-d4773ade6abe None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:12:02.343288 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78352d98-c00e-4b9e-bced-d4773ade6abe None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:12:02.343700 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78352d98-c00e-4b9e-bced-d4773ade6abe None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:12:02.344461 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78352d98-c00e-4b9e-bced-d4773ade6abe None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:12:02.346023 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78352d98-c00e-4b9e-bced-d4773ade6abe None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:12:02.353948 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b38a8c7-5870-4f1c-826b-fe458813b12e None None] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:12:02.354501 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b38a8c7-5870-4f1c-826b-fe458813b12e None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:12:02.354789 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b38a8c7-5870-4f1c-826b-fe458813b12e None None] PATH_INFO: `/` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:12:02.356393 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 462/921] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 12:12:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:12:02.357314 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application [None req-78352d98-c00e-4b9e-bced-d4773ade6abe None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 15 12:12:02.357314 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:12:02.357314 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 15 12:12:02.357314 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 15 12:12:02.357314 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 15 12:12:02.357314 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 15 12:12:02.357314 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 15 12:12:02.357314 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 15 12:12:02.357314 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 15 12:12:02.357314 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 15 12:12:02.357314 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 15 12:12:02.357314 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 15 12:12:02.357314 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:12:02.357314 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:12:02.357314 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:12:02.357314 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:12:02.357314 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:12:02.358486 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:12:02.358486 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:12:02.358486 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:12:02.358486 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:12:02.358486 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:12:02.358486 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:12:02.358486 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 15 12:12:02.358486 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 15 12:12:02.358486 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 15 12:12:02.358486 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 15 12:12:02.358486 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:12:02.358486 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:12:02.358486 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:12:02.358486 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:12:02.358486 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:12:02.358486 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:12:02.359514 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:12:02.359514 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application with Lock( Sep 15 12:12:02.359514 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:12:02.359514 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:12:02.359514 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:12:02.359514 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:12:02.359514 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:12:02.359514 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:12:02.359514 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:12:02.359514 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:12:02.359514 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 15 12:12:02.359514 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 15 12:12:02.359514 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 15 12:12:02.359514 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 15 12:12:02.362133 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9772fa70-2c10-4063-81b5-60dbd9fe0734 None None] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:12:02.362285 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 460/922] 10.222.0.15 () {56 vars in 1126 bytes} [Thu Sep 15 12:12:02 2022] GET /identity/v3/services/object-store => generated 93 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 15 12:12:02.362835 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9772fa70-2c10-4063-81b5-60dbd9fe0734 None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:12:02.363311 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9772fa70-2c10-4063-81b5-60dbd9fe0734 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:12:02.368319 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78352d98-c00e-4b9e-bced-d4773ade6abe None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:12:02.384326 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.auth.core [None req-9772fa70-2c10-4063-81b5-60dbd9fe0734 None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81449) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:12:02.386174 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-9772fa70-2c10-4063-81b5-60dbd9fe0734 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:12:02.476795 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:12:02.484872 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 463/923] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 12:12:02 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 15 12:12:02.493947 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78352d98-c00e-4b9e-bced-d4773ade6abe None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:12:02.545487 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93972dfb-879f-4921-9a81-a7630bd0a96f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:12:02.546123 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93972dfb-879f-4921-9a81-a7630bd0a96f None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:12:02.546407 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93972dfb-879f-4921-9a81-a7630bd0a96f None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:12:02.546682 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93972dfb-879f-4921-9a81-a7630bd0a96f None admin] PATH_INFO: `/v3/services` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:12:02.547323 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93972dfb-879f-4921-9a81-a7630bd0a96f None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:12:02.547624 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93972dfb-879f-4921-9a81-a7630bd0a96f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:12:02.549041 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93972dfb-879f-4921-9a81-a7630bd0a96f None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:12:02.554471 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 461/924] 10.222.0.15 () {56 vars in 1122 bytes} [Thu Sep 15 12:12:02 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:12:02.560141 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9772fa70-2c10-4063-81b5-60dbd9fe0734 None None] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:12:02.622073 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:12:02.663045 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9772fa70-2c10-4063-81b5-60dbd9fe0734 None None] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:12:02.699437 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9bd5ecd-f3cd-444c-8800-2d977c951ee2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:12:02.700066 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bd5ecd-f3cd-444c-8800-2d977c951ee2 None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:12:02.700341 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bd5ecd-f3cd-444c-8800-2d977c951ee2 None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:12:02.700602 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bd5ecd-f3cd-444c-8800-2d977c951ee2 None admin] PATH_INFO: `/v3/services` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:12:02.701294 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9bd5ecd-f3cd-444c-8800-2d977c951ee2 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81449) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:12:02.701591 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9bd5ecd-f3cd-444c-8800-2d977c951ee2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:12:02.702988 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9bd5ecd-f3cd-444c-8800-2d977c951ee2 None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:12:02.709864 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 464/925] 10.222.0.15 () {56 vars in 1122 bytes} [Thu Sep 15 12:12:02 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:12:02.715483 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93972dfb-879f-4921-9a81-a7630bd0a96f None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:12:02.766448 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9bd5ecd-f3cd-444c-8800-2d977c951ee2 None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:12:02.821952 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93972dfb-879f-4921-9a81-a7630bd0a96f None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:12:02.867901 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1b01f8-9cac-45fb-8880-515897ff588e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:12:02.868554 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1b01f8-9cac-45fb-8880-515897ff588e None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:12:02.868860 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1b01f8-9cac-45fb-8880-515897ff588e None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:12:02.869130 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1b01f8-9cac-45fb-8880-515897ff588e None admin] PATH_INFO: `/v3/endpoints` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:12:02.869824 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1b01f8-9cac-45fb-8880-515897ff588e None admin] RBAC: Adding query filter params (interface=public, service_id=33c672c5dd9342468ec3cf93dee68157, region_id=RegionOne) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:12:02.870127 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1b01f8-9cac-45fb-8880-515897ff588e None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:12:02.871633 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1b01f8-9cac-45fb-8880-515897ff588e None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:12:02.878810 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 462/926] 10.222.0.15 () {56 vars in 1251 bytes} [Thu Sep 15 12:12:02 2022] GET /identity/v3/endpoints?service_id=33c672c5dd9342468ec3cf93dee68157&interface=public®ion_id=RegionOne => generated 194 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:12:02.884003 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1b01f8-9cac-45fb-8880-515897ff588e None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:12:02.899222 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9bd5ecd-f3cd-444c-8800-2d977c951ee2 None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:12:02.910631 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:12:02.937820 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:12:02.952776 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8562f304-90c0-4d22-a52f-52940a3311cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:12:02.957944 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8562f304-90c0-4d22-a52f-52940a3311cc None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:12:02.958226 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8562f304-90c0-4d22-a52f-52940a3311cc None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:12:02.958495 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8562f304-90c0-4d22-a52f-52940a3311cc None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:12:02.959086 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8562f304-90c0-4d22-a52f-52940a3311cc None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:12:02.960455 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8562f304-90c0-4d22-a52f-52940a3311cc None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:12:02.973647 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application [None req-8562f304-90c0-4d22-a52f-52940a3311cc None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 15 12:12:02.973647 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:12:02.973647 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 15 12:12:02.973647 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 15 12:12:02.973647 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 15 12:12:02.973647 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 15 12:12:02.973647 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 15 12:12:02.973647 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 15 12:12:02.973647 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 15 12:12:02.973647 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application Sep 15 12:12:02.973647 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 15 12:12:02.973647 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application Sep 15 12:12:02.973647 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:12:02.973647 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:12:02.973647 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:12:02.973647 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:12:02.974608 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:12:02.974608 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:12:02.974608 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:12:02.974608 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:12:02.974608 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:12:02.974608 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:12:02.974608 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:12:02.974608 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 15 12:12:02.974608 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 15 12:12:02.974608 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 15 12:12:02.974608 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 15 12:12:02.974608 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:12:02.974608 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:12:02.974608 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:12:02.974608 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:12:02.974608 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:12:02.974608 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:12:02.975541 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:12:02.975541 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application with Lock( Sep 15 12:12:02.975541 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:12:02.975541 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:12:02.975541 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:12:02.975541 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:12:02.975541 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:12:02.975541 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:12:02.975541 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:12:02.975541 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:12:02.975541 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 15 12:12:02.975541 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 15 12:12:02.975541 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Sep 15 12:12:02.975541 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application Sep 15 12:12:02.975541 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 465/927] 10.222.0.15 () {56 vars in 1132 bytes} [Thu Sep 15 12:12:02 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 15 12:12:02.980663 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8562f304-90c0-4d22-a52f-52940a3311cc None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:12:03.030322 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1b01f8-9cac-45fb-8880-515897ff588e None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:12:03.076901 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f74bc0-66c6-45ad-b2e6-0173a7e851cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:12:03.077780 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f74bc0-66c6-45ad-b2e6-0173a7e851cb None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:12:03.078069 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f74bc0-66c6-45ad-b2e6-0173a7e851cb None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:12:03.078709 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f74bc0-66c6-45ad-b2e6-0173a7e851cb None admin] PATH_INFO: `/v3/services` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:12:03.079334 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f74bc0-66c6-45ad-b2e6-0173a7e851cb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:12:03.080725 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f74bc0-66c6-45ad-b2e6-0173a7e851cb None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:12:03.090283 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 463/928] 10.222.0.15 () {56 vars in 1087 bytes} [Thu Sep 15 12:12:02 2022] GET /identity/v3/services => generated 1497 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 12:12:03.096093 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8562f304-90c0-4d22-a52f-52940a3311cc None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:12:03.137563 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf2a929-1879-43ea-a687-41c3e034a300 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:12:03.138181 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf2a929-1879-43ea-a687-41c3e034a300 None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:12:03.138512 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf2a929-1879-43ea-a687-41c3e034a300 None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:12:03.138785 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf2a929-1879-43ea-a687-41c3e034a300 None admin] PATH_INFO: `/v3/services` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:12:03.139471 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf2a929-1879-43ea-a687-41c3e034a300 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81449) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:12:03.139774 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf2a929-1879-43ea-a687-41c3e034a300 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:12:03.141184 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf2a929-1879-43ea-a687-41c3e034a300 None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:12:03.147760 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 466/929] 10.222.0.15 () {56 vars in 1126 bytes} [Thu Sep 15 12:12:02 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:12:03.153866 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f74bc0-66c6-45ad-b2e6-0173a7e851cb None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:12:03.250025 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:12:03.272498 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f74bc0-66c6-45ad-b2e6-0173a7e851cb None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:12:03.307033 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfdc598-ed0f-4164-8742-d011d80fabd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:12:03.307569 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfdc598-ed0f-4164-8742-d011d80fabd2 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:12:03.307843 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfdc598-ed0f-4164-8742-d011d80fabd2 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:12:03.308108 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfdc598-ed0f-4164-8742-d011d80fabd2 None admin] PATH_INFO: `/v3/services` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:12:03.308722 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bfdc598-ed0f-4164-8742-d011d80fabd2 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:12:03.309052 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bfdc598-ed0f-4164-8742-d011d80fabd2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:12:03.310388 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bfdc598-ed0f-4164-8742-d011d80fabd2 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:12:03.315322 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 464/930] 10.222.0.15 () {56 vars in 1126 bytes} [Thu Sep 15 12:12:03 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:12:03.320807 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf2a929-1879-43ea-a687-41c3e034a300 None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:12:03.381353 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:12:03.429299 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf2a929-1879-43ea-a687-41c3e034a300 None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:12:03.464497 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7436e5f9-d6f6-42b3-87f1-c59e8ef310d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:12:03.465138 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7436e5f9-d6f6-42b3-87f1-c59e8ef310d6 None admin] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:12:03.465414 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7436e5f9-d6f6-42b3-87f1-c59e8ef310d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:12:03.465682 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7436e5f9-d6f6-42b3-87f1-c59e8ef310d6 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:12:03.466350 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7436e5f9-d6f6-42b3-87f1-c59e8ef310d6 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:12:03.467810 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7436e5f9-d6f6-42b3-87f1-c59e8ef310d6 None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:12:03.490286 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 467/931] 10.222.0.15 () {60 vars in 1144 bytes} [Thu Sep 15 12:12:03 2022] POST /identity/v3/endpoints => generated 347 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 12:12:04.072816 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ebf759-daac-4b3a-8629-8e1eb1b4c3e5 None None] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:12:04.073394 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ebf759-daac-4b3a-8629-8e1eb1b4c3e5 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:12:04.073394 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7ebf759-daac-4b3a-8629-8e1eb1b4c3e5 None None] PATH_INFO: `/` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:12:04.074665 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 465/932] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 12:12:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:12:04.080013 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab7b998-f9bc-46e6-a022-115fe6ee1d3d None None] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:12:04.080404 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab7b998-f9bc-46e6-a022-115fe6ee1d3d None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:12:04.080594 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab7b998-f9bc-46e6-a022-115fe6ee1d3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:12:04.099480 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.auth.core [None req-1ab7b998-f9bc-46e6-a022-115fe6ee1d3d None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81449) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:12:04.100658 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-1ab7b998-f9bc-46e6-a022-115fe6ee1d3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:12:04.176895 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 468/933] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 12:12:04 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 15 12:12:04.441710 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ebf759-daac-4b3a-8629-8e1eb1b4c3e5 None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:12:04.536340 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6beb41-6d4a-42fa-96c5-b778e5f970d4 None None] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:12:04.536669 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6beb41-6d4a-42fa-96c5-b778e5f970d4 None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:12:04.536938 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6beb41-6d4a-42fa-96c5-b778e5f970d4 None None] PATH_INFO: `/` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:12:04.539611 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 469/934] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 12:12:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:12:04.543504 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99148b07-4e6d-4832-a875-048680a852a7 None None] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:12:04.543784 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99148b07-4e6d-4832-a875-048680a852a7 None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:12:04.544061 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99148b07-4e6d-4832-a875-048680a852a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:12:04.547056 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7ebf759-daac-4b3a-8629-8e1eb1b4c3e5 None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:12:04.565419 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.auth.core [None req-99148b07-4e6d-4832-a875-048680a852a7 None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81449) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:12:04.567152 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-99148b07-4e6d-4832-a875-048680a852a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:12:04.590171 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9cdb0d7-fa4e-4170-ae30-02d65605462c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:12:04.590809 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cdb0d7-fa4e-4170-ae30-02d65605462c None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:12:04.591120 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cdb0d7-fa4e-4170-ae30-02d65605462c None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:12:04.591338 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cdb0d7-fa4e-4170-ae30-02d65605462c None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:12:04.602328 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9cdb0d7-fa4e-4170-ae30-02d65605462c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 12:12:04.605335 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:12:04.607664 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application [None req-b9cdb0d7-fa4e-4170-ae30-02d65605462c None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 15 12:12:04.607664 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:12:04.607664 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:12:04.607664 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:12:04.607664 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:12:04.607664 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:12:04.607664 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:12:04.607664 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:12:04.607664 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:12:04.607664 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:12:04.607664 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:12:04.607664 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:12:04.607664 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 12:12:04.607664 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 12:12:04.607664 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 12:12:04.607664 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 12:12:04.609738 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 12:12:04.609738 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 12:12:04.609738 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 12:12:04.609738 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 12:12:04.609738 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:12:04.609738 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:12:04.609738 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:12:04.609738 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:12:04.609738 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:12:04.609738 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:12:04.609738 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:12:04.609738 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application with Lock( Sep 15 12:12:04.609738 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:12:04.609738 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:12:04.609738 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:12:04.609738 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:12:04.610650 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:12:04.610650 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:12:04.610650 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:12:04.610650 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:12:04.610650 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 12:12:04.610650 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 12:12:04.610650 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 12:12:04.610650 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 12:12:04.610650 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 12:12:04.610650 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 12:12:04.610650 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 15 12:12:04.610650 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 15 12:12:04.610650 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 466/935] 10.222.0.15 () {56 vars in 1120 bytes} [Thu Sep 15 12:12:04 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:12:04.617918 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9cdb0d7-fa4e-4170-ae30-02d65605462c None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:12:04.630323 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:12:04.660994 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 470/936] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 12:12:04 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 15 12:12:04.732230 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9cdb0d7-fa4e-4170-ae30-02d65605462c None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:12:04.766293 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1fa749-2caa-4aa3-b07a-70f39e76d487 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:12:04.766909 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1fa749-2caa-4aa3-b07a-70f39e76d487 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:12:04.767191 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1fa749-2caa-4aa3-b07a-70f39e76d487 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:12:04.767459 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1fa749-2caa-4aa3-b07a-70f39e76d487 None admin] PATH_INFO: `/v3/roles` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:12:04.768122 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1fa749-2caa-4aa3-b07a-70f39e76d487 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:12:04.768426 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1fa749-2caa-4aa3-b07a-70f39e76d487 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:12:04.770365 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1fa749-2caa-4aa3-b07a-70f39e76d487 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:12:04.781898 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 467/937] 10.222.0.15 () {56 vars in 1115 bytes} [Thu Sep 15 12:12:04 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:12:04.789734 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99148b07-4e6d-4832-a875-048680a852a7 None None] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:12:04.891991 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99148b07-4e6d-4832-a875-048680a852a7 None None] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:12:04.927410 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42dd458d-69d8-4f70-8dd1-048a84c146da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:12:04.928042 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42dd458d-69d8-4f70-8dd1-048a84c146da None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:12:04.928321 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42dd458d-69d8-4f70-8dd1-048a84c146da None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:12:04.928592 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42dd458d-69d8-4f70-8dd1-048a84c146da None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:12:04.939237 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42dd458d-69d8-4f70-8dd1-048a84c146da None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:12:04.940779 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42dd458d-69d8-4f70-8dd1-048a84c146da None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:12:04.956771 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application [None req-42dd458d-69d8-4f70-8dd1-048a84c146da None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 12:12:04.956771 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:12:04.956771 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:12:04.956771 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:12:04.956771 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:12:04.956771 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:12:04.956771 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:12:04.956771 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:12:04.956771 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:12:04.956771 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:12:04.956771 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:12:04.956771 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:12:04.956771 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 12:12:04.956771 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 12:12:04.956771 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 12:12:04.956771 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 12:12:04.957742 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:12:04.957742 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:12:04.957742 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:12:04.957742 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:12:04.957742 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:12:04.957742 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:12:04.957742 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:12:04.957742 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application with Lock( Sep 15 12:12:04.957742 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:12:04.957742 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:12:04.957742 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:12:04.957742 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:12:04.957742 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:12:04.957742 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:12:04.957742 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:12:04.957742 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:12:04.957742 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 12:12:04.958942 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 12:12:04.958942 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 12:12:04.958942 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application Sep 15 12:12:04.958942 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 471/938] 10.222.0.15 () {56 vars in 1108 bytes} [Thu Sep 15 12:12:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 15 12:12:04.963585 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1fa749-2caa-4aa3-b07a-70f39e76d487 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:12:04.997270 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42dd458d-69d8-4f70-8dd1-048a84c146da None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:12:05.050774 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:12:05.053787 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:12:05.087836 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1fa749-2caa-4aa3-b07a-70f39e76d487 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:12:05.129455 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6914fc-3b5a-447d-b1d5-8a59f442fb24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:12:05.130075 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6914fc-3b5a-447d-b1d5-8a59f442fb24 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:12:05.130355 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6914fc-3b5a-447d-b1d5-8a59f442fb24 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:12:05.130625 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6914fc-3b5a-447d-b1d5-8a59f442fb24 None admin] PATH_INFO: `/v3/domains` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:12:05.131077 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42dd458d-69d8-4f70-8dd1-048a84c146da None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:12:05.131514 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd6914fc-3b5a-447d-b1d5-8a59f442fb24 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:12:05.131591 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd6914fc-3b5a-447d-b1d5-8a59f442fb24 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:12:05.132971 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd6914fc-3b5a-447d-b1d5-8a59f442fb24 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:12:05.153878 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 468/939] 10.222.0.15 () {56 vars in 1109 bytes} [Thu Sep 15 12:12:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:12:05.155600 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6914fc-3b5a-447d-b1d5-8a59f442fb24 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:12:05.181628 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4cc153-70c7-4693-b7d0-d6ab02c923c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:12:05.182636 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4cc153-70c7-4693-b7d0-d6ab02c923c8 None admin] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:12:05.183366 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4cc153-70c7-4693-b7d0-d6ab02c923c8 None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:12:05.183854 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4cc153-70c7-4693-b7d0-d6ab02c923c8 None admin] PATH_INFO: `/v3/projects` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:12:05.185483 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4cc153-70c7-4693-b7d0-d6ab02c923c8 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:12:05.187765 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4cc153-70c7-4693-b7d0-d6ab02c923c8 None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:12:05.192056 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: WARNING keystone.server.flask.common [None req-9a4cc153-70c7-4693-b7d0-d6ab02c923c8 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Sep 15 12:12:05.218409 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.sql.core [None req-9a4cc153-70c7-4693-b7d0-d6ab02c923c8 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 15 12:12:05.218409 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Sep 15 12:12:05.218409 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [parameters: {'id': 'bb0303d4fe73469d8616d097bebdd3c3', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Sep 15 12:12:05.218409 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81449) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 15 12:12:05.232061 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application [None req-9a4cc153-70c7-4693-b7d0-d6ab02c923c8 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id bb0303d4fe73469d8616d097bebdd3c3.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id bb0303d4fe73469d8616d097bebdd3c3. Sep 15 12:12:05.232061 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:12:05.232061 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 15 12:12:05.232061 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 15 12:12:05.232061 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 15 12:12:05.232061 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 15 12:12:05.232061 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 15 12:12:05.232061 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application result = self._query(query) Sep 15 12:12:05.232061 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 15 12:12:05.232061 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application conn.query(q) Sep 15 12:12:05.232061 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 15 12:12:05.232061 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 15 12:12:05.232061 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 15 12:12:05.233084 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application result.read() Sep 15 12:12:05.233084 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 15 12:12:05.233084 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 15 12:12:05.233084 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 15 12:12:05.233084 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 15 12:12:05.233084 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 15 12:12:05.233084 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 15 12:12:05.233084 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 15 12:12:05.233084 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 15 12:12:05.233084 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 15 12:12:05.233084 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application Sep 15 12:12:05.233084 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 15 12:12:05.233084 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application Sep 15 12:12:05.233084 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:12:05.233084 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 15 12:12:05.233084 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 15 12:12:05.233084 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Sep 15 12:12:05.233084 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return project_ref.to_dict() Sep 15 12:12:05.234261 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 15 12:12:05.234261 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application next(self.gen) Sep 15 12:12:05.234261 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 15 12:12:05.234261 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application yield resource Sep 15 12:12:05.234261 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 15 12:12:05.234261 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application next(self.gen) Sep 15 12:12:05.234261 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 15 12:12:05.234261 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application self.session.rollback() Sep 15 12:12:05.234261 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 15 12:12:05.234261 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application self.force_reraise() Sep 15 12:12:05.234261 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 15 12:12:05.234261 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application raise self.value Sep 15 12:12:05.234261 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 15 12:12:05.234261 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 15 12:12:05.234261 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 15 12:12:05.234261 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application session.commit() Sep 15 12:12:05.234261 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 15 12:12:05.235373 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 15 12:12:05.235373 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 15 12:12:05.235373 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application self._prepare_impl() Sep 15 12:12:05.235373 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 15 12:12:05.235373 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application self.session.flush() Sep 15 12:12:05.235373 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 15 12:12:05.235373 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application self._flush(objects) Sep 15 12:12:05.235373 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 15 12:12:05.235373 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 15 12:12:05.235373 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 15 12:12:05.235373 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application compat.raise_( Sep 15 12:12:05.235373 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 15 12:12:05.235373 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application raise exception Sep 15 12:12:05.235373 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 15 12:12:05.235373 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application flush_context.execute() Sep 15 12:12:05.235373 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 15 12:12:05.235373 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application rec.execute(self) Sep 15 12:12:05.236466 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 15 12:12:05.236466 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 15 12:12:05.236466 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 15 12:12:05.236466 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 15 12:12:05.236466 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Sep 15 12:12:05.236466 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application c = connection._execute_20( Sep 15 12:12:05.236466 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 15 12:12:05.236466 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 15 12:12:05.236466 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 15 12:12:05.236466 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 15 12:12:05.236466 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 15 12:12:05.236466 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 15 12:12:05.236466 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 15 12:12:05.236466 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 15 12:12:05.236466 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 15 12:12:05.237535 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 15 12:12:05.237535 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 15 12:12:05.237535 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application raise exception Sep 15 12:12:05.237535 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 15 12:12:05.237535 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 15 12:12:05.237535 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 15 12:12:05.237535 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 15 12:12:05.237535 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 15 12:12:05.237535 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application result = self._query(query) Sep 15 12:12:05.237535 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 15 12:12:05.237535 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application conn.query(q) Sep 15 12:12:05.237535 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 15 12:12:05.237535 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 15 12:12:05.237535 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 15 12:12:05.237535 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application result.read() Sep 15 12:12:05.237535 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 15 12:12:05.237535 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 15 12:12:05.238667 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 15 12:12:05.238667 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 15 12:12:05.238667 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 15 12:12:05.238667 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 15 12:12:05.238667 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 15 12:12:05.238667 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 15 12:12:05.238667 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Sep 15 12:12:05.238667 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Sep 15 12:12:05.238667 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application [parameters: {'id': 'bb0303d4fe73469d8616d097bebdd3c3', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Sep 15 12:12:05.238667 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 15 12:12:05.238667 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application Sep 15 12:12:05.238667 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 15 12:12:05.238667 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application Sep 15 12:12:05.238667 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:12:05.238667 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Sep 15 12:12:05.239656 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Sep 15 12:12:05.239656 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 15 12:12:05.239656 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 15 12:12:05.239656 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Sep 15 12:12:05.239656 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application Sep 15 12:12:05.239656 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 15 12:12:05.239656 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application Sep 15 12:12:05.239656 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:12:05.239656 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:12:05.239656 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:12:05.239656 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:12:05.239656 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:12:05.239656 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:12:05.239656 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:12:05.239656 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:12:05.239656 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:12:05.239656 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:12:05.241294 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:12:05.241294 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Sep 15 12:12:05.241294 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Sep 15 12:12:05.241294 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:12:05.241294 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:12:05.241294 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Sep 15 12:12:05.241294 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application raise exception.Conflict( Sep 15 12:12:05.241294 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id bb0303d4fe73469d8616d097bebdd3c3. Sep 15 12:12:05.241294 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application Sep 15 12:12:05.244139 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 472/940] 10.222.0.15 () {60 vars in 1140 bytes} [Thu Sep 15 12:12:04 2022] POST /identity/v3/projects => generated 262 bytes in 247 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Sep 15 12:12:05.252292 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4cc153-70c7-4693-b7d0-d6ab02c923c8 None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:12:05.289708 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6914fc-3b5a-447d-b1d5-8a59f442fb24 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:12:05.327597 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:12:05.336478 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5dd714-d318-4693-8118-125f8b0c6750 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:12:05.337703 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5dd714-d318-4693-8118-125f8b0c6750 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:12:05.338170 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5dd714-d318-4693-8118-125f8b0c6750 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:12:05.338743 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5dd714-d318-4693-8118-125f8b0c6750 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:12:05.345614 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:12:05.347773 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5dd714-d318-4693-8118-125f8b0c6750 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:12:05.349533 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5dd714-d318-4693-8118-125f8b0c6750 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:12:05.365582 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application [None req-8a5dd714-d318-4693-8118-125f8b0c6750 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 15 12:12:05.365582 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:12:05.365582 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:12:05.365582 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:12:05.365582 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:12:05.365582 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:12:05.365582 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:12:05.365582 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:12:05.365582 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:12:05.365582 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:12:05.365582 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:12:05.365582 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:12:05.365582 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 15 12:12:05.365582 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 15 12:12:05.365582 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 15 12:12:05.365582 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 15 12:12:05.366590 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:12:05.366590 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:12:05.366590 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 15 12:12:05.366590 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 12:12:05.366590 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 15 12:12:05.366590 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 12:12:05.366590 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:12:05.366590 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:12:05.366590 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:12:05.366590 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:12:05.366590 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:12:05.366590 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application with Lock( Sep 15 12:12:05.366590 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:12:05.366590 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:12:05.366590 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:12:05.366590 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:12:05.366590 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:12:05.366590 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:12:05.367558 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:12:05.367558 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:12:05.367558 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 15 12:12:05.367558 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 15 12:12:05.367558 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 15 12:12:05.367558 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 15 12:12:05.367558 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 15 12:12:05.367558 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 15 12:12:05.367558 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 15 12:12:05.367558 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 15 12:12:05.375113 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 469/941] 10.222.0.15 () {56 vars in 1117 bytes} [Thu Sep 15 12:12:05 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:12:05.376902 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5dd714-d318-4693-8118-125f8b0c6750 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:12:05.380244 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4cc153-70c7-4693-b7d0-d6ab02c923c8 None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:12:05.417988 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c973bf-5b75-4418-8ad3-96b009294f4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:12:05.418633 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c973bf-5b75-4418-8ad3-96b009294f4a None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:12:05.418915 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c973bf-5b75-4418-8ad3-96b009294f4a None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:12:05.419212 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c973bf-5b75-4418-8ad3-96b009294f4a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:12:05.428061 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c973bf-5b75-4418-8ad3-96b009294f4a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:12:05.429744 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c973bf-5b75-4418-8ad3-96b009294f4a None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:12:05.441623 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application [None req-c7c973bf-5b75-4418-8ad3-96b009294f4a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 12:12:05.441623 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:12:05.441623 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:12:05.441623 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:12:05.441623 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:12:05.441623 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:12:05.441623 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:12:05.441623 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:12:05.441623 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:12:05.441623 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:12:05.441623 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:12:05.441623 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:12:05.441623 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 15 12:12:05.441623 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 15 12:12:05.441623 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 15 12:12:05.442503 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 15 12:12:05.442503 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:12:05.442503 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:12:05.442503 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:12:05.442503 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:12:05.442503 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:12:05.442503 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:12:05.442503 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:12:05.442503 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application with Lock( Sep 15 12:12:05.442503 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:12:05.442503 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:12:05.442503 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:12:05.442503 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:12:05.442503 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:12:05.442503 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:12:05.442503 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:12:05.442503 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:12:05.443447 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 15 12:12:05.443447 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 15 12:12:05.443447 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 15 12:12:05.443447 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 15 12:12:05.443447 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 15 12:12:05.443447 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 15 12:12:05.443447 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 12:12:05.443447 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application Sep 15 12:12:05.447362 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 473/942] 10.222.0.15 () {56 vars in 1111 bytes} [Thu Sep 15 12:12:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:12:05.448619 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6df8c0-81f2-4e4b-b5fb-00e56daf0fe2 None None] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:12:05.448932 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6df8c0-81f2-4e4b-b5fb-00e56daf0fe2 None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:12:05.449206 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6df8c0-81f2-4e4b-b5fb-00e56daf0fe2 None None] PATH_INFO: `/` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:12:05.453791 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 474/943] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 12:12:05 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:12:05.454152 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6df8c0-81f2-4e4b-b5fb-00e56daf0fe2 None None] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:12:05.488858 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5dd714-d318-4693-8118-125f8b0c6750 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:12:05.523532 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a1a52c9-dbd6-4e9c-9cc6-809502028f0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:12:05.524073 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1a52c9-dbd6-4e9c-9cc6-809502028f0b None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:12:05.524354 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1a52c9-dbd6-4e9c-9cc6-809502028f0b None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:12:05.524640 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a1a52c9-dbd6-4e9c-9cc6-809502028f0b None admin] PATH_INFO: `/v3/users` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:12:05.525376 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a1a52c9-dbd6-4e9c-9cc6-809502028f0b None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:12:05.525969 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a1a52c9-dbd6-4e9c-9cc6-809502028f0b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:12:05.527089 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a1a52c9-dbd6-4e9c-9cc6-809502028f0b None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:12:05.551699 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 470/944] 10.222.0.15 () {56 vars in 1149 bytes} [Thu Sep 15 12:12:05 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:12:05.551699 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c277ca2-254f-4613-9622-a993c5d79af1 None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:12:05.551699 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c277ca2-254f-4613-9622-a993c5d79af1 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:12:05.551699 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c277ca2-254f-4613-9622-a993c5d79af1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:12:05.565484 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6df8c0-81f2-4e4b-b5fb-00e56daf0fe2 None None] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:12:05.570377 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-2c277ca2-254f-4613-9622-a993c5d79af1 None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:12:05.571625 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-2c277ca2-254f-4613-9622-a993c5d79af1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:12:05.602205 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6c5bd3-25b7-4f0c-a922-3ec2f8bea0b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:12:05.602810 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6c5bd3-25b7-4f0c-a922-3ec2f8bea0b8 None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:12:05.603088 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6c5bd3-25b7-4f0c-a922-3ec2f8bea0b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:12:05.603355 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a6c5bd3-25b7-4f0c-a922-3ec2f8bea0b8 None admin] PATH_INFO: `/v3/projects` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:12:05.604002 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6c5bd3-25b7-4f0c-a922-3ec2f8bea0b8 None admin] RBAC: Adding query filter params (name=service) {{(pid=81449) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:12:05.604572 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6c5bd3-25b7-4f0c-a922-3ec2f8bea0b8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:12:05.606005 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a6c5bd3-25b7-4f0c-a922-3ec2f8bea0b8 None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:12:05.607504 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:12:05.621599 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:12:05.636787 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 475/945] 10.222.0.15 () {56 vars in 1112 bytes} [Thu Sep 15 12:12:05 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:12:05.636787 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6c5bd3-25b7-4f0c-a922-3ec2f8bea0b8 None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:12:05.651201 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 471/946] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 12:12:05 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 15 12:12:05.749758 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05728b9-cd3c-4c5f-8335-f2fb062186b1 None None] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:12:05.750059 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05728b9-cd3c-4c5f-8335-f2fb062186b1 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:12:05.750304 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05728b9-cd3c-4c5f-8335-f2fb062186b1 None None] PATH_INFO: `/` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:12:05.751909 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 472/947] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 12:12:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 15 12:12:05.755196 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a6c5bd3-25b7-4f0c-a922-3ec2f8bea0b8 None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:12:05.757925 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7602f725-6157-4921-9eb3-66efb7d999d6 None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:12:05.758210 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7602f725-6157-4921-9eb3-66efb7d999d6 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:12:05.758483 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7602f725-6157-4921-9eb3-66efb7d999d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:12:05.778526 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-7602f725-6157-4921-9eb3-66efb7d999d6 None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:12:05.779657 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-7602f725-6157-4921-9eb3-66efb7d999d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:12:05.798568 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d4706c-10c2-4dbb-aecc-7001ee63aeb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:12:05.799232 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d4706c-10c2-4dbb-aecc-7001ee63aeb5 None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:12:05.799815 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d4706c-10c2-4dbb-aecc-7001ee63aeb5 None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:12:05.800117 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d4706c-10c2-4dbb-aecc-7001ee63aeb5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:12:05.814510 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d4706c-10c2-4dbb-aecc-7001ee63aeb5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:12:05.817184 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d4706c-10c2-4dbb-aecc-7001ee63aeb5 None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:12:05.841794 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application [None req-30d4706c-10c2-4dbb-aecc-7001ee63aeb5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 12:12:05.841794 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:12:05.841794 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:12:05.841794 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:12:05.841794 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:12:05.841794 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:12:05.841794 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:12:05.841794 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:12:05.841794 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:12:05.841794 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:12:05.841794 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:12:05.841794 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:12:05.841794 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 12:12:05.841794 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 12:12:05.841794 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 12:12:05.841794 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 12:12:05.842786 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:12:05.842786 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:12:05.842786 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:12:05.842786 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:12:05.842786 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:12:05.842786 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:12:05.842786 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:12:05.842786 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application with Lock( Sep 15 12:12:05.842786 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:12:05.842786 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:12:05.842786 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:12:05.842786 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:12:05.842786 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:12:05.842786 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:12:05.842786 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:12:05.842786 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:12:05.842786 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 12:12:05.843830 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 12:12:05.843830 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 12:12:05.843830 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application Sep 15 12:12:05.843830 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 476/948] 10.222.0.15 () {56 vars in 1108 bytes} [Thu Sep 15 12:12:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:12:05.848706 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d4706c-10c2-4dbb-aecc-7001ee63aeb5 None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:12:05.874864 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 473/949] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 12:12:05 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 15 12:12:05.900773 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:12:05.936521 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7602f725-6157-4921-9eb3-66efb7d999d6 None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:12:05.965890 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d4706c-10c2-4dbb-aecc-7001ee63aeb5 None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:12:06.005736 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4981f82-ca12-4e97-8d1d-b34376649c5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:12:06.006273 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4981f82-ca12-4e97-8d1d-b34376649c5a None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:12:06.006553 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4981f82-ca12-4e97-8d1d-b34376649c5a None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:12:06.006820 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4981f82-ca12-4e97-8d1d-b34376649c5a None admin] PATH_INFO: `/v3/domains` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:12:06.007469 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4981f82-ca12-4e97-8d1d-b34376649c5a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81449) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:12:06.007757 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4981f82-ca12-4e97-8d1d-b34376649c5a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:12:06.009120 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4981f82-ca12-4e97-8d1d-b34376649c5a None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:12:06.022699 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 477/950] 10.222.0.15 () {56 vars in 1109 bytes} [Thu Sep 15 12:12:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 12:12:06.027791 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4981f82-ca12-4e97-8d1d-b34376649c5a None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:12:06.071998 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7602f725-6157-4921-9eb3-66efb7d999d6 None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:12:06.084275 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:12:06.114898 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22a4c77-d9eb-428e-93ea-4c19a572dc55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:12:06.116487 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22a4c77-d9eb-428e-93ea-4c19a572dc55 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:12:06.117116 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22a4c77-d9eb-428e-93ea-4c19a572dc55 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:12:06.117620 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22a4c77-d9eb-428e-93ea-4c19a572dc55 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:12:06.118409 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22a4c77-d9eb-428e-93ea-4c19a572dc55 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:12:06.120274 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22a4c77-d9eb-428e-93ea-4c19a572dc55 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:12:06.131609 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application [None req-d22a4c77-d9eb-428e-93ea-4c19a572dc55 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 15 12:12:06.131609 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:12:06.131609 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 15 12:12:06.131609 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 15 12:12:06.131609 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 15 12:12:06.131609 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 15 12:12:06.131609 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 15 12:12:06.131609 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 15 12:12:06.131609 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 15 12:12:06.131609 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 15 12:12:06.131609 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 15 12:12:06.131609 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 15 12:12:06.131609 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:12:06.131609 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:12:06.131609 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:12:06.131609 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:12:06.131609 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:12:06.132609 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:12:06.132609 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:12:06.132609 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:12:06.132609 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:12:06.132609 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:12:06.132609 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:12:06.132609 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 15 12:12:06.132609 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 15 12:12:06.132609 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 15 12:12:06.132609 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 15 12:12:06.132609 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:12:06.132609 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:12:06.132609 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:12:06.132609 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:12:06.132609 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:12:06.132609 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:12:06.133494 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:12:06.133494 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application with Lock( Sep 15 12:12:06.133494 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:12:06.133494 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:12:06.133494 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:12:06.133494 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:12:06.133494 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:12:06.133494 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:12:06.133494 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:12:06.133494 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:12:06.133494 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 15 12:12:06.133494 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 15 12:12:06.133494 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 15 12:12:06.133494 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 15 12:12:06.141705 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 474/951] 10.222.0.15 () {56 vars in 1126 bytes} [Thu Sep 15 12:12:05 2022] GET /identity/v3/services/object-store => generated 93 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 15 12:12:06.143386 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22a4c77-d9eb-428e-93ea-4c19a572dc55 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:12:06.147158 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4981f82-ca12-4e97-8d1d-b34376649c5a None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:12:06.156203 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:12:06.192670 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71d9b57-be6f-4a82-bad4-18b18cedca8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:12:06.193410 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71d9b57-be6f-4a82-bad4-18b18cedca8c None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:12:06.193693 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71d9b57-be6f-4a82-bad4-18b18cedca8c None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:12:06.193964 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71d9b57-be6f-4a82-bad4-18b18cedca8c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:12:06.201567 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71d9b57-be6f-4a82-bad4-18b18cedca8c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:12:06.203005 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71d9b57-be6f-4a82-bad4-18b18cedca8c None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:12:06.214479 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application [None req-b71d9b57-be6f-4a82-bad4-18b18cedca8c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 12:12:06.214479 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:12:06.214479 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:12:06.214479 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:12:06.214479 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:12:06.214479 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:12:06.214479 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:12:06.214479 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:12:06.214479 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:12:06.214479 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:12:06.214479 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:12:06.214479 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:12:06.214479 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 15 12:12:06.214479 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 15 12:12:06.214479 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 15 12:12:06.215405 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 15 12:12:06.215405 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:12:06.215405 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:12:06.215405 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:12:06.215405 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:12:06.215405 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:12:06.215405 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:12:06.215405 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:12:06.215405 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application with Lock( Sep 15 12:12:06.215405 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:12:06.215405 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:12:06.215405 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:12:06.215405 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:12:06.215405 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:12:06.215405 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:12:06.215405 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:12:06.215405 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:12:06.216303 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 15 12:12:06.216303 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 15 12:12:06.216303 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 15 12:12:06.216303 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 15 12:12:06.216303 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 15 12:12:06.216303 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 15 12:12:06.216303 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 12:12:06.216303 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application Sep 15 12:12:06.216303 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 478/952] 10.222.0.15 () {56 vars in 1111 bytes} [Thu Sep 15 12:12:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:12:06.216998 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71d9b57-be6f-4a82-bad4-18b18cedca8c None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:12:06.261054 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22a4c77-d9eb-428e-93ea-4c19a572dc55 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:12:06.296639 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560f07d4-d38b-4b86-9b2f-c24bbdf4aade None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:12:06.297275 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560f07d4-d38b-4b86-9b2f-c24bbdf4aade None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:12:06.297554 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560f07d4-d38b-4b86-9b2f-c24bbdf4aade None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:12:06.297824 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560f07d4-d38b-4b86-9b2f-c24bbdf4aade None admin] PATH_INFO: `/v3/services` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:12:06.298447 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560f07d4-d38b-4b86-9b2f-c24bbdf4aade None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:12:06.298749 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560f07d4-d38b-4b86-9b2f-c24bbdf4aade None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:12:06.300647 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-560f07d4-d38b-4b86-9b2f-c24bbdf4aade None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:12:06.305630 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 475/953] 10.222.0.15 () {56 vars in 1122 bytes} [Thu Sep 15 12:12:06 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:12:06.309997 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560f07d4-d38b-4b86-9b2f-c24bbdf4aade None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:12:06.327791 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71d9b57-be6f-4a82-bad4-18b18cedca8c None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:12:06.347843 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:12:06.373741 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3baa4907-aa35-4987-8327-0e9c6fbd63d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:12:06.374520 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3baa4907-aa35-4987-8327-0e9c6fbd63d3 None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:12:06.374801 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3baa4907-aa35-4987-8327-0e9c6fbd63d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:12:06.375320 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3baa4907-aa35-4987-8327-0e9c6fbd63d3 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:12:06.376297 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3baa4907-aa35-4987-8327-0e9c6fbd63d3 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:12:06.377965 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3baa4907-aa35-4987-8327-0e9c6fbd63d3 None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:12:06.387975 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application [None req-3baa4907-aa35-4987-8327-0e9c6fbd63d3 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 15 12:12:06.387975 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:12:06.387975 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 15 12:12:06.387975 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 15 12:12:06.387975 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 15 12:12:06.387975 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 15 12:12:06.387975 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 15 12:12:06.387975 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 15 12:12:06.387975 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 15 12:12:06.387975 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application Sep 15 12:12:06.387975 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 15 12:12:06.387975 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application Sep 15 12:12:06.387975 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:12:06.387975 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:12:06.387975 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:12:06.387975 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:12:06.387975 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:12:06.392895 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:12:06.392895 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:12:06.392895 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:12:06.392895 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:12:06.392895 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:12:06.392895 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:12:06.392895 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 15 12:12:06.392895 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 15 12:12:06.392895 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 15 12:12:06.392895 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 15 12:12:06.392895 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:12:06.392895 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:12:06.392895 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:12:06.392895 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:12:06.392895 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:12:06.392895 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:12:06.394470 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:12:06.394470 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application with Lock( Sep 15 12:12:06.394470 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:12:06.394470 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:12:06.394470 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:12:06.394470 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:12:06.394470 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:12:06.394470 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:12:06.394470 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:12:06.394470 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:12:06.394470 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 15 12:12:06.394470 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 15 12:12:06.394470 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 15 12:12:06.394470 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application Sep 15 12:12:06.394470 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 479/954] 10.222.0.15 () {56 vars in 1111 bytes} [Thu Sep 15 12:12:06 2022] GET /identity/v3/services/compute => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:12:06.395393 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3baa4907-aa35-4987-8327-0e9c6fbd63d3 None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:12:06.421126 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560f07d4-d38b-4b86-9b2f-c24bbdf4aade None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:12:06.422286 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:12:06.455492 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9051c30e-e05e-42a6-9038-fe2d05035b91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:12:06.456140 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9051c30e-e05e-42a6-9038-fe2d05035b91 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:12:06.456976 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9051c30e-e05e-42a6-9038-fe2d05035b91 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:12:06.457242 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9051c30e-e05e-42a6-9038-fe2d05035b91 None admin] PATH_INFO: `/v3/projects` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:12:06.457939 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9051c30e-e05e-42a6-9038-fe2d05035b91 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:12:06.458243 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9051c30e-e05e-42a6-9038-fe2d05035b91 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:12:06.459662 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9051c30e-e05e-42a6-9038-fe2d05035b91 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:12:06.471749 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 476/955] 10.222.0.15 () {56 vars in 1148 bytes} [Thu Sep 15 12:12:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:12:06.473193 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9051c30e-e05e-42a6-9038-fe2d05035b91 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:12:06.501919 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3baa4907-aa35-4987-8327-0e9c6fbd63d3 None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:12:06.541683 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4e6994-2d1f-4f15-9001-968f13de90e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:12:06.542573 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4e6994-2d1f-4f15-9001-968f13de90e9 None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:12:06.543001 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4e6994-2d1f-4f15-9001-968f13de90e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:12:06.543280 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4e6994-2d1f-4f15-9001-968f13de90e9 None admin] PATH_INFO: `/v3/services` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:12:06.543905 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4e6994-2d1f-4f15-9001-968f13de90e9 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81449) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:12:06.544214 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4e6994-2d1f-4f15-9001-968f13de90e9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:12:06.545626 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4e6994-2d1f-4f15-9001-968f13de90e9 None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:12:06.550566 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 480/956] 10.222.0.15 () {56 vars in 1122 bytes} [Thu Sep 15 12:12:06 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:12:06.551328 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4e6994-2d1f-4f15-9001-968f13de90e9 None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:12:06.584134 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:12:06.590729 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9051c30e-e05e-42a6-9038-fe2d05035b91 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:12:06.629411 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ddb78e3-bb88-4187-a64f-8839f5e06779 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:12:06.630268 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddb78e3-bb88-4187-a64f-8839f5e06779 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:12:06.630640 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddb78e3-bb88-4187-a64f-8839f5e06779 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:12:06.631018 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddb78e3-bb88-4187-a64f-8839f5e06779 None admin] PATH_INFO: `/v3/services` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:12:06.631749 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ddb78e3-bb88-4187-a64f-8839f5e06779 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:12:06.632155 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ddb78e3-bb88-4187-a64f-8839f5e06779 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:12:06.633642 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ddb78e3-bb88-4187-a64f-8839f5e06779 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:12:06.643720 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 477/957] 10.222.0.15 () {56 vars in 1112 bytes} [Thu Sep 15 12:12:06 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:12:06.643720 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ddb78e3-bb88-4187-a64f-8839f5e06779 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:12:06.657741 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:12:06.665690 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4e6994-2d1f-4f15-9001-968f13de90e9 None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:12:06.701178 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049bef37-2f73-4552-892c-9c116b9fc737 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:12:06.701934 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049bef37-2f73-4552-892c-9c116b9fc737 None admin] REQUEST_METHOD: `PUT` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:12:06.702283 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049bef37-2f73-4552-892c-9c116b9fc737 None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:12:06.702621 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049bef37-2f73-4552-892c-9c116b9fc737 None admin] PATH_INFO: `/v3/projects/540a32ef88764e8c8d9201ae22f6ddaf/users/527f4dd6031546c785bc6360dacb60af/roles/c0320da65e0b4cdd8924c20a99b978af` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:12:06.704997 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-049bef37-2f73-4552-892c-9c116b9fc737 None admin] RBAC: Authorizing `identity:create_grant(project_id=540a32ef88764e8c8d9201ae22f6ddaf, user_id=527f4dd6031546c785bc6360dacb60af, role_id=c0320da65e0b4cdd8924c20a99b978af)` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:12:06.707094 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-049bef37-2f73-4552-892c-9c116b9fc737 None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:12:06.720335 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 481/958] 10.222.0.15 () {58 vars in 1439 bytes} [Thu Sep 15 12:12:06 2022] PUT /identity/v3/projects/540a32ef88764e8c8d9201ae22f6ddaf/users/527f4dd6031546c785bc6360dacb60af/roles/c0320da65e0b4cdd8924c20a99b978af => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:12:06.721468 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049bef37-2f73-4552-892c-9c116b9fc737 None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:12:06.772720 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ddb78e3-bb88-4187-a64f-8839f5e06779 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:12:06.825580 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20bff3ce-4423-449d-b7d7-16ecd1e9a0c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:12:06.826581 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bff3ce-4423-449d-b7d7-16ecd1e9a0c1 None admin] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:12:06.827010 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bff3ce-4423-449d-b7d7-16ecd1e9a0c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:12:06.827423 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bff3ce-4423-449d-b7d7-16ecd1e9a0c1 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:12:06.829027 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20bff3ce-4423-449d-b7d7-16ecd1e9a0c1 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:12:06.831174 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20bff3ce-4423-449d-b7d7-16ecd1e9a0c1 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:12:06.869295 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 478/959] 10.222.0.15 () {60 vars in 1144 bytes} [Thu Sep 15 12:12:06 2022] POST /identity/v3/endpoints => generated 349 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 12:12:06.871633 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049bef37-2f73-4552-892c-9c116b9fc737 None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:12:06.923707 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb402b1-8884-4158-9304-93ed42a21e7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:12:06.924337 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb402b1-8884-4158-9304-93ed42a21e7d None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:12:06.924621 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb402b1-8884-4158-9304-93ed42a21e7d None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:12:06.925461 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb402b1-8884-4158-9304-93ed42a21e7d None admin] PATH_INFO: `/v3/services` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:12:06.926103 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb402b1-8884-4158-9304-93ed42a21e7d None admin] RBAC: Adding query filter params (type=compute) {{(pid=81449) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:12:06.926402 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb402b1-8884-4158-9304-93ed42a21e7d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:12:06.927904 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb402b1-8884-4158-9304-93ed42a21e7d None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:12:06.933620 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 482/960] 10.222.0.15 () {56 vars in 1112 bytes} [Thu Sep 15 12:12:06 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:12:07.957511 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d527f7d1-8313-4ee4-8885-3d33c8c4d1a2 None None] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:12:07.958031 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d527f7d1-8313-4ee4-8885-3d33c8c4d1a2 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:12:07.958031 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d527f7d1-8313-4ee4-8885-3d33c8c4d1a2 None None] PATH_INFO: `/` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:12:07.959374 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 479/961] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 12:12:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:12:07.964658 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8709dcbb-ad2a-41d0-85d4-99cc7b974bf7 None None] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:12:07.965065 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8709dcbb-ad2a-41d0-85d4-99cc7b974bf7 None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:12:07.965233 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8709dcbb-ad2a-41d0-85d4-99cc7b974bf7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:12:07.985975 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.auth.core [None req-8709dcbb-ad2a-41d0-85d4-99cc7b974bf7 None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81449) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:12:07.988172 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-8709dcbb-ad2a-41d0-85d4-99cc7b974bf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:12:07.998283 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:12:08.060906 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 483/962] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 12:12:07 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 15 12:12:08.326423 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d527f7d1-8313-4ee4-8885-3d33c8c4d1a2 None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:12:08.345592 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:12:08.443662 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d527f7d1-8313-4ee4-8885-3d33c8c4d1a2 None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:12:08.477820 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22cbd29b-a9a7-4b03-8940-3b500eb6f8e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:12:08.478433 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cbd29b-a9a7-4b03-8940-3b500eb6f8e3 None admin] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:12:08.478711 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cbd29b-a9a7-4b03-8940-3b500eb6f8e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:12:08.479229 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cbd29b-a9a7-4b03-8940-3b500eb6f8e3 None admin] PATH_INFO: `/v3/users` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:12:08.479881 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22cbd29b-a9a7-4b03-8940-3b500eb6f8e3 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:12:08.481334 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22cbd29b-a9a7-4b03-8940-3b500eb6f8e3 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:12:08.483834 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: WARNING keystone.server.flask.common [None req-22cbd29b-a9a7-4b03-8940-3b500eb6f8e3 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Sep 15 12:12:08.499196 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.sql.core [None req-22cbd29b-a9a7-4b03-8940-3b500eb6f8e3 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 15 12:12:08.499196 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Sep 15 12:12:08.499196 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [parameters: {'user_id': 'deaaac1d6c92415ca235df89a4bd057f', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Sep 15 12:12:08.499196 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81448) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Sep 15 12:12:08.517311 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application [None req-22cbd29b-a9a7-4b03-8940-3b500eb6f8e3 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Sep 15 12:12:08.517311 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:12:08.517311 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 15 12:12:08.517311 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 15 12:12:08.517311 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 15 12:12:08.517311 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 15 12:12:08.517311 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 15 12:12:08.517311 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application result = self._query(query) Sep 15 12:12:08.517311 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 15 12:12:08.517311 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application conn.query(q) Sep 15 12:12:08.517311 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 15 12:12:08.517311 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 15 12:12:08.517311 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 15 12:12:08.517311 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application result.read() Sep 15 12:12:08.517311 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 15 12:12:08.518283 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 15 12:12:08.518283 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 15 12:12:08.518283 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 15 12:12:08.518283 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 15 12:12:08.518283 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 15 12:12:08.518283 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 15 12:12:08.518283 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 15 12:12:08.518283 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 15 12:12:08.518283 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 15 12:12:08.518283 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Sep 15 12:12:08.518283 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 15 12:12:08.518283 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:12:08.518283 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Sep 15 12:12:08.518283 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return method(*args, **kwargs) Sep 15 12:12:08.518283 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Sep 15 12:12:08.518283 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Sep 15 12:12:08.518283 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 15 12:12:08.518283 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application next(self.gen) Sep 15 12:12:08.519346 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Sep 15 12:12:08.519346 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application yield resource Sep 15 12:12:08.519346 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Sep 15 12:12:08.519346 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application next(self.gen) Sep 15 12:12:08.519346 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Sep 15 12:12:08.519346 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application self.session.rollback() Sep 15 12:12:08.519346 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Sep 15 12:12:08.519346 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application self.force_reraise() Sep 15 12:12:08.519346 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Sep 15 12:12:08.519346 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application raise self.value Sep 15 12:12:08.519346 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Sep 15 12:12:08.519346 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Sep 15 12:12:08.519346 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Sep 15 12:12:08.519346 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application session.commit() Sep 15 12:12:08.519346 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Sep 15 12:12:08.519346 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Sep 15 12:12:08.519346 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Sep 15 12:12:08.521073 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application self._prepare_impl() Sep 15 12:12:08.521073 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Sep 15 12:12:08.521073 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application self.session.flush() Sep 15 12:12:08.521073 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Sep 15 12:12:08.521073 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application self._flush(objects) Sep 15 12:12:08.521073 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Sep 15 12:12:08.521073 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Sep 15 12:12:08.521073 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Sep 15 12:12:08.521073 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application compat.raise_( Sep 15 12:12:08.521073 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 15 12:12:08.521073 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application raise exception Sep 15 12:12:08.521073 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Sep 15 12:12:08.521073 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application flush_context.execute() Sep 15 12:12:08.521073 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Sep 15 12:12:08.521073 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application rec.execute(self) Sep 15 12:12:08.521073 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Sep 15 12:12:08.521073 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Sep 15 12:12:08.522157 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Sep 15 12:12:08.522157 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application _emit_insert_statements( Sep 15 12:12:08.522157 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Sep 15 12:12:08.522157 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application result = connection._execute_20( Sep 15 12:12:08.522157 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Sep 15 12:12:08.522157 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Sep 15 12:12:08.522157 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Sep 15 12:12:08.522157 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Sep 15 12:12:08.522157 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Sep 15 12:12:08.522157 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application ret = self._execute_context( Sep 15 12:12:08.522157 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Sep 15 12:12:08.522157 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Sep 15 12:12:08.522157 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Sep 15 12:12:08.522157 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Sep 15 12:12:08.522157 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Sep 15 12:12:08.522157 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application raise exception Sep 15 12:12:08.523259 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Sep 15 12:12:08.523259 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application self.dialect.do_execute( Sep 15 12:12:08.523259 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Sep 15 12:12:08.523259 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Sep 15 12:12:08.523259 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Sep 15 12:12:08.523259 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application result = self._query(query) Sep 15 12:12:08.523259 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Sep 15 12:12:08.523259 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application conn.query(q) Sep 15 12:12:08.523259 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Sep 15 12:12:08.523259 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Sep 15 12:12:08.523259 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Sep 15 12:12:08.523259 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application result.read() Sep 15 12:12:08.523259 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Sep 15 12:12:08.523259 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Sep 15 12:12:08.523259 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Sep 15 12:12:08.523259 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application packet.raise_for_error() Sep 15 12:12:08.525926 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Sep 15 12:12:08.525926 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Sep 15 12:12:08.525926 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Sep 15 12:12:08.525926 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Sep 15 12:12:08.525926 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Sep 15 12:12:08.525926 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Sep 15 12:12:08.525926 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application [parameters: {'user_id': 'deaaac1d6c92415ca235df89a4bd057f', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Sep 15 12:12:08.525926 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Sep 15 12:12:08.525926 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 15 12:12:08.525926 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 15 12:12:08.525926 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 15 12:12:08.525926 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:12:08.525926 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:12:08.525926 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:12:08.525926 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:12:08.526862 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:12:08.526862 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:12:08.526862 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:12:08.526862 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:12:08.526862 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:12:08.526862 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:12:08.526862 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:12:08.526862 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Sep 15 12:12:08.526862 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Sep 15 12:12:08.526862 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:12:08.526862 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:12:08.526862 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 15 12:12:08.526862 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 12:12:08.526862 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 15 12:12:08.526862 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 12:12:08.526862 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Sep 15 12:12:08.526862 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Sep 15 12:12:08.527862 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Sep 15 12:12:08.527862 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Sep 15 12:12:08.527862 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Sep 15 12:12:08.527862 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Sep 15 12:12:08.527862 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Sep 15 12:12:08.527862 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 15 12:12:08.527862 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 480/963] 10.222.0.15 () {60 vars in 1131 bytes} [Thu Sep 15 12:12:08 2022] POST /identity/v3/users => generated 160 bytes in 193 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:12:08.530159 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8709dcbb-ad2a-41d0-85d4-99cc7b974bf7 None None] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:12:08.642453 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8709dcbb-ad2a-41d0-85d4-99cc7b974bf7 None None] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:12:08.678341 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09a24d8-aced-4e93-b896-a879db5d9c02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:12:08.678988 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09a24d8-aced-4e93-b896-a879db5d9c02 None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:12:08.679268 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09a24d8-aced-4e93-b896-a879db5d9c02 None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:12:08.679540 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09a24d8-aced-4e93-b896-a879db5d9c02 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:12:08.687365 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09a24d8-aced-4e93-b896-a879db5d9c02 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:12:08.688898 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09a24d8-aced-4e93-b896-a879db5d9c02 None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:12:08.701759 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application [None req-f09a24d8-aced-4e93-b896-a879db5d9c02 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Sep 15 12:12:08.701759 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:12:08.701759 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:12:08.701759 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:12:08.701759 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:12:08.701759 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:12:08.701759 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:12:08.701759 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:12:08.701759 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:12:08.701759 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:12:08.701759 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:12:08.701759 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:12:08.701759 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 15 12:12:08.701759 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 15 12:12:08.701759 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 15 12:12:08.701759 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 15 12:12:08.702882 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:12:08.702882 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:12:08.702882 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 15 12:12:08.702882 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 12:12:08.702882 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 15 12:12:08.702882 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 12:12:08.702882 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:12:08.702882 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:12:08.702882 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:12:08.702882 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:12:08.702882 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:12:08.702882 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application with Lock( Sep 15 12:12:08.702882 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:12:08.702882 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:12:08.702882 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:12:08.702882 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:12:08.702882 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:12:08.702882 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:12:08.703840 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:12:08.703840 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:12:08.703840 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 15 12:12:08.703840 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 15 12:12:08.703840 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 15 12:12:08.703840 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 15 12:12:08.703840 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 15 12:12:08.703840 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 15 12:12:08.703840 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Sep 15 12:12:08.703840 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application Sep 15 12:12:08.703840 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 484/964] 10.222.0.15 () {56 vars in 1099 bytes} [Thu Sep 15 12:12:08 2022] GET /identity/v3/users/cinder => generated 84 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:12:08.709396 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22cbd29b-a9a7-4b03-8940-3b500eb6f8e3 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:12:08.813062 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:12:08.841243 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22cbd29b-a9a7-4b03-8940-3b500eb6f8e3 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:12:08.879389 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6127569-d341-42cc-8eb4-4f462618a735 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:12:08.880502 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6127569-d341-42cc-8eb4-4f462618a735 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:12:08.881014 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6127569-d341-42cc-8eb4-4f462618a735 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:12:08.881489 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6127569-d341-42cc-8eb4-4f462618a735 None admin] PATH_INFO: `/v3/users` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:12:08.882407 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6127569-d341-42cc-8eb4-4f462618a735 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:12:08.882910 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6127569-d341-42cc-8eb4-4f462618a735 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:12:08.884777 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6127569-d341-42cc-8eb4-4f462618a735 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:12:08.927193 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 481/965] 10.222.0.15 () {56 vars in 1101 bytes} [Thu Sep 15 12:12:08 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:12:09.086168 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb31e863-b0c5-4b1c-abe1-592b4f410d07 None None] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:12:09.087259 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb31e863-b0c5-4b1c-abe1-592b4f410d07 None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:12:09.087745 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb31e863-b0c5-4b1c-abe1-592b4f410d07 None None] PATH_INFO: `/` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:12:09.090013 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 485/966] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 12:12:09 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:12:09.097910 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65633f1b-ea3d-46ce-aa34-2a10d7ac9fda None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:12:09.098965 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65633f1b-ea3d-46ce-aa34-2a10d7ac9fda None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:12:09.099449 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65633f1b-ea3d-46ce-aa34-2a10d7ac9fda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:12:09.121351 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-65633f1b-ea3d-46ce-aa34-2a10d7ac9fda None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:12:09.123284 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-65633f1b-ea3d-46ce-aa34-2a10d7ac9fda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:12:09.190845 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5cc749-09f7-4067-837f-66dbc900ed07 None None] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:12:09.191189 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5cc749-09f7-4067-837f-66dbc900ed07 None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:12:09.191402 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5cc749-09f7-4067-837f-66dbc900ed07 None None] PATH_INFO: `/` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:12:09.193015 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 486/967] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 12:12:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:12:09.197224 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a65dd0-429b-4ede-8339-8943efc46f38 None None] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:12:09.197506 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a65dd0-429b-4ede-8339-8943efc46f38 None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:12:09.197779 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a65dd0-429b-4ede-8339-8943efc46f38 None None] PATH_INFO: `/` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:12:09.199397 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 487/968] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 12:12:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 15 12:12:09.200539 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa1d83d-c244-4b69-8b9e-e0b9a05b814c None None] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:12:09.200862 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa1d83d-c244-4b69-8b9e-e0b9a05b814c None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:12:09.201122 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa1d83d-c244-4b69-8b9e-e0b9a05b814c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:12:09.202825 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 482/969] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 12:12:09 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 15 12:12:09.205451 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a6a67f-e269-412b-a938-6f74858fe81a None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:12:09.205741 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a6a67f-e269-412b-a938-6f74858fe81a None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:12:09.206016 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a6a67f-e269-412b-a938-6f74858fe81a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:12:09.212468 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:12:09.229979 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.auth.core [None req-4aa1d83d-c244-4b69-8b9e-e0b9a05b814c None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81449) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:12:09.231183 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-14a6a67f-e269-412b-a938-6f74858fe81a None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:12:09.231336 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-4aa1d83d-c244-4b69-8b9e-e0b9a05b814c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:12:09.232434 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-14a6a67f-e269-412b-a938-6f74858fe81a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:12:09.306848 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:12:09.316665 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 483/970] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 12:12:09 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 15 12:12:09.328792 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 488/971] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 12:12:09 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 15 12:12:09.491138 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a6a67f-e269-412b-a938-6f74858fe81a None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:12:09.586551 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa1d83d-c244-4b69-8b9e-e0b9a05b814c None None] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:12:09.590630 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a6a67f-e269-412b-a938-6f74858fe81a None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:12:09.625304 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba70766b-3bc4-4398-939e-29c3baec985d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:12:09.625941 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba70766b-3bc4-4398-939e-29c3baec985d None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:12:09.626218 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba70766b-3bc4-4398-939e-29c3baec985d None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:12:09.626488 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba70766b-3bc4-4398-939e-29c3baec985d None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:12:09.636432 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba70766b-3bc4-4398-939e-29c3baec985d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 12:12:09.641219 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application [None req-ba70766b-3bc4-4398-939e-29c3baec985d None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 15 12:12:09.641219 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:12:09.641219 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:12:09.641219 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:12:09.641219 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:12:09.641219 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:12:09.641219 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:12:09.641219 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:12:09.641219 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:12:09.641219 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:12:09.641219 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:12:09.641219 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:12:09.641219 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 12:12:09.641219 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 12:12:09.641219 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 12:12:09.641219 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 12:12:09.642133 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 12:12:09.642133 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 12:12:09.642133 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 12:12:09.642133 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 12:12:09.642133 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:12:09.642133 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:12:09.642133 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:12:09.642133 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:12:09.642133 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:12:09.642133 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:12:09.642133 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:12:09.642133 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application with Lock( Sep 15 12:12:09.642133 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:12:09.642133 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:12:09.642133 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:12:09.642133 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:12:09.642991 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:12:09.642991 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:12:09.642991 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:12:09.642991 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:12:09.642991 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 12:12:09.642991 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 12:12:09.642991 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 12:12:09.642991 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 12:12:09.642991 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 12:12:09.642991 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 12:12:09.642991 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 15 12:12:09.642991 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 15 12:12:09.642991 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 484/972] 10.222.0.15 () {56 vars in 1120 bytes} [Thu Sep 15 12:12:09 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:12:09.643764 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba70766b-3bc4-4398-939e-29c3baec985d None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:12:09.693046 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa1d83d-c244-4b69-8b9e-e0b9a05b814c None None] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:12:09.699281 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:12:09.727224 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5929a1a7-c8ed-45fc-9709-cada62bb8883 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:12:09.727827 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5929a1a7-c8ed-45fc-9709-cada62bb8883 None admin] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:12:09.728103 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5929a1a7-c8ed-45fc-9709-cada62bb8883 None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:12:09.728378 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5929a1a7-c8ed-45fc-9709-cada62bb8883 None admin] PATH_INFO: `/v3/services` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:12:09.729314 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5929a1a7-c8ed-45fc-9709-cada62bb8883 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:12:09.730705 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5929a1a7-c8ed-45fc-9709-cada62bb8883 None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:12:09.734051 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:12:09.743546 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba70766b-3bc4-4398-939e-29c3baec985d None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:12:09.745025 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 489/973] 10.222.0.15 () {60 vars in 1141 bytes} [Thu Sep 15 12:12:09 2022] POST /identity/v3/services => generated 242 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 15 12:12:09.745666 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5929a1a7-c8ed-45fc-9709-cada62bb8883 None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:12:09.796782 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1892560c-ffb0-4afd-8ab1-7d8840cad527 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:12:09.797475 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1892560c-ffb0-4afd-8ab1-7d8840cad527 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:12:09.797757 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1892560c-ffb0-4afd-8ab1-7d8840cad527 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:12:09.798028 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1892560c-ffb0-4afd-8ab1-7d8840cad527 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:12:09.798629 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1892560c-ffb0-4afd-8ab1-7d8840cad527 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:12:09.800107 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1892560c-ffb0-4afd-8ab1-7d8840cad527 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:12:09.811163 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application [None req-1892560c-ffb0-4afd-8ab1-7d8840cad527 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 15 12:12:09.811163 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:12:09.811163 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 15 12:12:09.811163 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 15 12:12:09.811163 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 15 12:12:09.811163 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 15 12:12:09.811163 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 15 12:12:09.811163 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 15 12:12:09.811163 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 15 12:12:09.811163 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 15 12:12:09.811163 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 15 12:12:09.811163 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 15 12:12:09.811163 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:12:09.811163 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:12:09.811163 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:12:09.811163 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:12:09.811163 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:12:09.812446 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:12:09.812446 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:12:09.812446 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:12:09.812446 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:12:09.812446 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:12:09.812446 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:12:09.812446 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 15 12:12:09.812446 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 15 12:12:09.812446 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 15 12:12:09.812446 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 15 12:12:09.812446 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:12:09.812446 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:12:09.812446 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:12:09.812446 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:12:09.812446 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:12:09.812446 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:12:09.813482 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:12:09.813482 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application with Lock( Sep 15 12:12:09.813482 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:12:09.813482 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:12:09.813482 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:12:09.813482 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:12:09.813482 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:12:09.813482 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:12:09.813482 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:12:09.813482 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:12:09.813482 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 15 12:12:09.813482 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 15 12:12:09.813482 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 15 12:12:09.813482 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 15 12:12:09.814351 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 485/974] 10.222.0.15 () {56 vars in 1126 bytes} [Thu Sep 15 12:12:09 2022] GET /identity/v3/services/object-store => generated 93 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:12:09.820456 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1892560c-ffb0-4afd-8ab1-7d8840cad527 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:12:09.867924 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5929a1a7-c8ed-45fc-9709-cada62bb8883 None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:12:09.908368 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d068c663-9cc5-4059-8317-04da8816a9f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:12:09.909020 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d068c663-9cc5-4059-8317-04da8816a9f5 None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:12:09.909373 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d068c663-9cc5-4059-8317-04da8816a9f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:12:09.909645 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d068c663-9cc5-4059-8317-04da8816a9f5 None admin] PATH_INFO: `/v3/roles` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:12:09.910764 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d068c663-9cc5-4059-8317-04da8816a9f5 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81449) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:12:09.911593 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d068c663-9cc5-4059-8317-04da8816a9f5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:12:09.912842 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d068c663-9cc5-4059-8317-04da8816a9f5 None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:12:09.921741 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 490/975] 10.222.0.15 () {56 vars in 1115 bytes} [Thu Sep 15 12:12:09 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:12:09.928217 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d068c663-9cc5-4059-8317-04da8816a9f5 None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:12:09.938541 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1892560c-ffb0-4afd-8ab1-7d8840cad527 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:12:09.961045 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:12:09.985771 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79bff01a-9e1e-4b7e-97b7-39c43add846b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:12:09.986473 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bff01a-9e1e-4b7e-97b7-39c43add846b None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:12:09.986851 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bff01a-9e1e-4b7e-97b7-39c43add846b None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:12:09.987203 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bff01a-9e1e-4b7e-97b7-39c43add846b None admin] PATH_INFO: `/v3/services` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:12:09.988286 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:12:09.988450 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79bff01a-9e1e-4b7e-97b7-39c43add846b None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:12:09.988735 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79bff01a-9e1e-4b7e-97b7-39c43add846b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:12:09.990116 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79bff01a-9e1e-4b7e-97b7-39c43add846b None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:12:09.995694 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 486/976] 10.222.0.15 () {56 vars in 1122 bytes} [Thu Sep 15 12:12:09 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 12:12:10.001273 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79bff01a-9e1e-4b7e-97b7-39c43add846b None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:12:10.036556 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d068c663-9cc5-4059-8317-04da8816a9f5 None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:12:10.073626 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9ebb50-d518-4be0-9dd5-2f1077a8f1b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:12:10.074286 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9ebb50-d518-4be0-9dd5-2f1077a8f1b7 None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:12:10.074547 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9ebb50-d518-4be0-9dd5-2f1077a8f1b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:12:10.074821 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9ebb50-d518-4be0-9dd5-2f1077a8f1b7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:12:10.086686 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9ebb50-d518-4be0-9dd5-2f1077a8f1b7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:12:10.088125 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9ebb50-d518-4be0-9dd5-2f1077a8f1b7 None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:12:10.102357 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application [None req-ca9ebb50-d518-4be0-9dd5-2f1077a8f1b7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 12:12:10.102357 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:12:10.102357 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:12:10.102357 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:12:10.102357 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:12:10.102357 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:12:10.102357 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:12:10.102357 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:12:10.102357 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:12:10.102357 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:12:10.102357 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:12:10.102357 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:12:10.102357 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 12:12:10.102357 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 12:12:10.102357 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 12:12:10.102357 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 12:12:10.103331 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:12:10.103331 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:12:10.103331 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:12:10.103331 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:12:10.103331 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:12:10.103331 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:12:10.103331 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:12:10.103331 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application with Lock( Sep 15 12:12:10.103331 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:12:10.103331 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:12:10.103331 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:12:10.103331 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:12:10.103331 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:12:10.103331 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:12:10.103331 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:12:10.103331 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:12:10.103331 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 12:12:10.104520 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 12:12:10.104520 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 12:12:10.104520 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application Sep 15 12:12:10.104520 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 491/977] 10.222.0.15 () {56 vars in 1108 bytes} [Thu Sep 15 12:12:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 15 12:12:10.109288 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9ebb50-d518-4be0-9dd5-2f1077a8f1b7 None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:12:10.111365 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79bff01a-9e1e-4b7e-97b7-39c43add846b None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:12:10.150984 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52060692-3686-4a70-80cd-4271ba34fde1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:12:10.152062 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52060692-3686-4a70-80cd-4271ba34fde1 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:12:10.152342 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52060692-3686-4a70-80cd-4271ba34fde1 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:12:10.152607 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52060692-3686-4a70-80cd-4271ba34fde1 None admin] PATH_INFO: `/v3/services` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:12:10.153827 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52060692-3686-4a70-80cd-4271ba34fde1 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:12:10.154130 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52060692-3686-4a70-80cd-4271ba34fde1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:12:10.156222 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52060692-3686-4a70-80cd-4271ba34fde1 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:12:10.164909 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 487/978] 10.222.0.15 () {56 vars in 1122 bytes} [Thu Sep 15 12:12:10 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 12:12:10.168187 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52060692-3686-4a70-80cd-4271ba34fde1 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:12:10.216504 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9ebb50-d518-4be0-9dd5-2f1077a8f1b7 None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:12:10.220036 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:12:10.226208 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:12:10.253905 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64238ef2-6144-4305-8674-cb77e2522830 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:12:10.254429 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64238ef2-6144-4305-8674-cb77e2522830 None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:12:10.255120 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64238ef2-6144-4305-8674-cb77e2522830 None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:12:10.255402 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64238ef2-6144-4305-8674-cb77e2522830 None admin] PATH_INFO: `/v3/domains` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:12:10.256042 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64238ef2-6144-4305-8674-cb77e2522830 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81449) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:12:10.256345 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64238ef2-6144-4305-8674-cb77e2522830 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:12:10.258088 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64238ef2-6144-4305-8674-cb77e2522830 None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:12:10.271768 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 492/979] 10.222.0.15 () {56 vars in 1109 bytes} [Thu Sep 15 12:12:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 12:12:10.277040 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64238ef2-6144-4305-8674-cb77e2522830 None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:12:10.280006 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52060692-3686-4a70-80cd-4271ba34fde1 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:12:10.314969 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b9a04d-52ea-4b16-9384-91a9342f4d6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:12:10.315629 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b9a04d-52ea-4b16-9384-91a9342f4d6a None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:12:10.315907 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b9a04d-52ea-4b16-9384-91a9342f4d6a None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:12:10.316178 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b9a04d-52ea-4b16-9384-91a9342f4d6a None admin] PATH_INFO: `/v3/endpoints` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:12:10.316907 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b9a04d-52ea-4b16-9384-91a9342f4d6a None admin] RBAC: Adding query filter params (interface=admin, service_id=33c672c5dd9342468ec3cf93dee68157, region_id=RegionOne) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:12:10.317214 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b9a04d-52ea-4b16-9384-91a9342f4d6a None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:12:10.318557 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b9a04d-52ea-4b16-9384-91a9342f4d6a None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:12:10.328438 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 488/980] 10.222.0.15 () {56 vars in 1249 bytes} [Thu Sep 15 12:12:10 2022] GET /identity/v3/endpoints?service_id=33c672c5dd9342468ec3cf93dee68157&interface=admin®ion_id=RegionOne => generated 193 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 12:12:10.329340 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b9a04d-52ea-4b16-9384-91a9342f4d6a None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:12:10.379197 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64238ef2-6144-4305-8674-cb77e2522830 None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:12:10.415499 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc1c812-8964-4822-8bd6-3c6f51e67630 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:12:10.416430 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc1c812-8964-4822-8bd6-3c6f51e67630 None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:12:10.416891 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc1c812-8964-4822-8bd6-3c6f51e67630 None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:12:10.417162 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc1c812-8964-4822-8bd6-3c6f51e67630 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:12:10.424344 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc1c812-8964-4822-8bd6-3c6f51e67630 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:12:10.426419 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc1c812-8964-4822-8bd6-3c6f51e67630 None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:12:10.438033 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application [None req-4fc1c812-8964-4822-8bd6-3c6f51e67630 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 15 12:12:10.438033 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:12:10.438033 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:12:10.438033 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:12:10.438033 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:12:10.438033 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:12:10.438033 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:12:10.438033 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:12:10.438033 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:12:10.438033 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:12:10.438033 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:12:10.438033 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:12:10.438033 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 15 12:12:10.438033 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 15 12:12:10.438033 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 15 12:12:10.438033 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 15 12:12:10.439152 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:12:10.439152 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:12:10.439152 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 15 12:12:10.439152 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 12:12:10.439152 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 15 12:12:10.439152 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 12:12:10.439152 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:12:10.439152 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:12:10.439152 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:12:10.439152 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:12:10.439152 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:12:10.439152 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application with Lock( Sep 15 12:12:10.439152 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:12:10.439152 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:12:10.439152 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:12:10.439152 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:12:10.439152 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:12:10.439152 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:12:10.445195 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:12:10.445195 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:12:10.445195 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 15 12:12:10.445195 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 15 12:12:10.445195 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 15 12:12:10.445195 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 15 12:12:10.445195 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 15 12:12:10.445195 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 15 12:12:10.445195 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Sep 15 12:12:10.445195 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application Sep 15 12:12:10.445873 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b9a04d-52ea-4b16-9384-91a9342f4d6a None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:12:10.445974 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 493/981] 10.222.0.15 () {56 vars in 1117 bytes} [Thu Sep 15 12:12:10 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 15 12:12:10.445974 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc1c812-8964-4822-8bd6-3c6f51e67630 None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:12:10.454004 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:12:10.477621 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:12:10.483009 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf0fe55-9615-40e1-9365-c942a8a5ad45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:12:10.483631 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf0fe55-9615-40e1-9365-c942a8a5ad45 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:12:10.483907 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf0fe55-9615-40e1-9365-c942a8a5ad45 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:12:10.484171 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf0fe55-9615-40e1-9365-c942a8a5ad45 None admin] PATH_INFO: `/v3/services` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:12:10.484967 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf0fe55-9615-40e1-9365-c942a8a5ad45 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:12:10.487161 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf0fe55-9615-40e1-9365-c942a8a5ad45 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:12:10.495318 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 489/982] 10.222.0.15 () {56 vars in 1087 bytes} [Thu Sep 15 12:12:10 2022] GET /identity/v3/services => generated 1727 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 12:12:10.567531 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc1c812-8964-4822-8bd6-3c6f51e67630 None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:12:10.605402 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de1bacd-d972-4581-ac24-0063335829f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:12:10.606120 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de1bacd-d972-4581-ac24-0063335829f3 None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:12:10.606468 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de1bacd-d972-4581-ac24-0063335829f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:12:10.606807 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de1bacd-d972-4581-ac24-0063335829f3 None admin] PATH_INFO: `/v3/users` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:12:10.607613 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de1bacd-d972-4581-ac24-0063335829f3 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81449) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:12:10.607988 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de1bacd-d972-4581-ac24-0063335829f3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:12:10.609519 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de1bacd-d972-4581-ac24-0063335829f3 None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:12:10.634187 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 494/983] 10.222.0.15 () {56 vars in 1149 bytes} [Thu Sep 15 12:12:10 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 12:12:10.635759 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf0fe55-9615-40e1-9365-c942a8a5ad45 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:12:10.742381 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf0fe55-9615-40e1-9365-c942a8a5ad45 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:12:10.774744 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debf41d8-0777-424e-83d3-08afd88a623a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:12:10.775508 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debf41d8-0777-424e-83d3-08afd88a623a None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:12:10.775890 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debf41d8-0777-424e-83d3-08afd88a623a None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:12:10.776265 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debf41d8-0777-424e-83d3-08afd88a623a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:12:10.816184 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:12:10.847548 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-debf41d8-0777-424e-83d3-08afd88a623a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:12:10.849108 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-debf41d8-0777-424e-83d3-08afd88a623a None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:12:10.952623 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 490/984] 10.222.0.15 () {58 vars in 1282 bytes} [Thu Sep 15 12:12:10 2022] GET /identity/v3/auth/tokens => generated 1570 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 15 12:12:10.954563 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de1bacd-d972-4581-ac24-0063335829f3 None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:12:11.047352 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:12:11.063785 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de1bacd-d972-4581-ac24-0063335829f3 None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:12:11.096496 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3dfe45f-3d66-4a74-aca8-3a37992eb555 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:12:11.097707 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3dfe45f-3d66-4a74-aca8-3a37992eb555 None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:12:11.097942 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3dfe45f-3d66-4a74-aca8-3a37992eb555 None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:12:11.098192 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3dfe45f-3d66-4a74-aca8-3a37992eb555 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:12:11.108159 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3dfe45f-3d66-4a74-aca8-3a37992eb555 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:12:11.109594 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3dfe45f-3d66-4a74-aca8-3a37992eb555 None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:12:11.127546 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application [None req-c3dfe45f-3d66-4a74-aca8-3a37992eb555 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 12:12:11.127546 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:12:11.127546 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:12:11.127546 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:12:11.127546 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:12:11.127546 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:12:11.127546 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:12:11.127546 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:12:11.127546 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:12:11.127546 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:12:11.127546 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:12:11.127546 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:12:11.127546 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 12:12:11.127546 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 12:12:11.127546 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 12:12:11.127546 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 12:12:11.128527 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:12:11.128527 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:12:11.128527 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:12:11.128527 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:12:11.128527 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:12:11.128527 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:12:11.128527 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:12:11.128527 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application with Lock( Sep 15 12:12:11.128527 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:12:11.128527 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:12:11.128527 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:12:11.128527 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:12:11.128527 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:12:11.128527 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:12:11.128527 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:12:11.128527 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:12:11.128527 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 12:12:11.130113 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 12:12:11.130113 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 12:12:11.130113 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application Sep 15 12:12:11.132842 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 495/985] 10.222.0.15 () {56 vars in 1108 bytes} [Thu Sep 15 12:12:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:12:11.134314 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debf41d8-0777-424e-83d3-08afd88a623a None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:12:11.203608 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:12:11.234730 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debf41d8-0777-424e-83d3-08afd88a623a None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:12:11.269479 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6398c3a-6572-40b5-8f76-f8cc1bb2e2ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:12:11.270088 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6398c3a-6572-40b5-8f76-f8cc1bb2e2ec None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:12:11.270359 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6398c3a-6572-40b5-8f76-f8cc1bb2e2ec None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:12:11.270621 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6398c3a-6572-40b5-8f76-f8cc1bb2e2ec None admin] PATH_INFO: `/v3/domains` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:12:11.271250 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6398c3a-6572-40b5-8f76-f8cc1bb2e2ec None admin] RBAC: Adding query filter params (name=Default) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:12:11.271547 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6398c3a-6572-40b5-8f76-f8cc1bb2e2ec None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:12:11.272921 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6398c3a-6572-40b5-8f76-f8cc1bb2e2ec None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:12:11.284052 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 491/986] 10.222.0.15 () {56 vars in 1109 bytes} [Thu Sep 15 12:12:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:12:11.289489 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3dfe45f-3d66-4a74-aca8-3a37992eb555 None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:12:11.390026 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3dfe45f-3d66-4a74-aca8-3a37992eb555 None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:12:11.425291 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9f08ef-22e7-411a-bca2-9896d5525105 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:12:11.425920 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9f08ef-22e7-411a-bca2-9896d5525105 None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:12:11.426197 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9f08ef-22e7-411a-bca2-9896d5525105 None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:12:11.426467 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9f08ef-22e7-411a-bca2-9896d5525105 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:12:11.432917 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c9f08ef-22e7-411a-bca2-9896d5525105 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:12:11.434337 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c9f08ef-22e7-411a-bca2-9896d5525105 None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:12:11.436009 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:12:11.447916 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application [None req-9c9f08ef-22e7-411a-bca2-9896d5525105 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 12:12:11.447916 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:12:11.447916 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:12:11.447916 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:12:11.447916 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:12:11.447916 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:12:11.447916 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:12:11.447916 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:12:11.447916 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:12:11.447916 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:12:11.447916 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:12:11.447916 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:12:11.447916 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 15 12:12:11.447916 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 15 12:12:11.447916 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 15 12:12:11.449103 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 15 12:12:11.449103 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:12:11.449103 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:12:11.449103 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:12:11.449103 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:12:11.449103 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:12:11.449103 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:12:11.449103 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:12:11.449103 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application with Lock( Sep 15 12:12:11.449103 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:12:11.449103 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:12:11.449103 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:12:11.449103 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:12:11.449103 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:12:11.449103 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:12:11.449103 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:12:11.449103 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:12:11.450030 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 15 12:12:11.450030 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 15 12:12:11.450030 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 15 12:12:11.450030 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 15 12:12:11.450030 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 15 12:12:11.450030 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 15 12:12:11.450030 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 12:12:11.450030 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application Sep 15 12:12:11.450030 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 496/987] 10.222.0.15 () {56 vars in 1111 bytes} [Thu Sep 15 12:12:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:12:11.456287 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6398c3a-6572-40b5-8f76-f8cc1bb2e2ec None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:12:11.556149 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6398c3a-6572-40b5-8f76-f8cc1bb2e2ec None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:12:11.585349 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:12:11.591045 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fb22e2-744d-41aa-80c3-871b4018335e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:12:11.591646 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fb22e2-744d-41aa-80c3-871b4018335e None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:12:11.591922 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fb22e2-744d-41aa-80c3-871b4018335e None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:12:11.592186 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fb22e2-744d-41aa-80c3-871b4018335e None admin] PATH_INFO: `/v3/projects` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:12:11.592945 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27fb22e2-744d-41aa-80c3-871b4018335e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:12:11.593247 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27fb22e2-744d-41aa-80c3-871b4018335e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:12:11.594645 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27fb22e2-744d-41aa-80c3-871b4018335e None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:12:11.607004 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 492/988] 10.222.0.15 () {56 vars in 1148 bytes} [Thu Sep 15 12:12:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:12:11.613391 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9f08ef-22e7-411a-bca2-9896d5525105 None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:12:11.713941 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9f08ef-22e7-411a-bca2-9896d5525105 None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:12:11.747557 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a5b457-6650-45c8-a29a-0d28da781688 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:12:11.748161 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a5b457-6650-45c8-a29a-0d28da781688 None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:12:11.748434 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a5b457-6650-45c8-a29a-0d28da781688 None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:12:11.748698 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a5b457-6650-45c8-a29a-0d28da781688 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:12:11.749441 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a5b457-6650-45c8-a29a-0d28da781688 None admin] RBAC: Adding query filter params (role.id=c0320da65e0b4cdd8924c20a99b978af, scope.project.id=540a32ef88764e8c8d9201ae22f6ddaf, user.id=527f4dd6031546c785bc6360dacb60af) {{(pid=81449) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:12:11.749746 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a5b457-6650-45c8-a29a-0d28da781688 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:12:11.751155 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a5b457-6650-45c8-a29a-0d28da781688 None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:12:11.757791 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 497/989] 10.222.0.15 () {56 vars in 1374 bytes} [Thu Sep 15 12:12:11 2022] GET /identity/v3/role_assignments?user.id=527f4dd6031546c785bc6360dacb60af&scope.project.id=540a32ef88764e8c8d9201ae22f6ddaf&role.id=c0320da65e0b4cdd8924c20a99b978af => generated 608 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 12:12:11.938926 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9b6dc6-6843-47ae-98dd-5f778075d4e6 None None] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:12:11.939203 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9b6dc6-6843-47ae-98dd-5f778075d4e6 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:12:11.939470 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9b6dc6-6843-47ae-98dd-5f778075d4e6 None None] PATH_INFO: `/` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:12:11.941637 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 493/990] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 12:12:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:12:11.946916 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad573d7a-5a6b-4f0c-98a4-f1e3eeafd7da None None] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:12:11.947202 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad573d7a-5a6b-4f0c-98a4-f1e3eeafd7da None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:12:11.947474 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad573d7a-5a6b-4f0c-98a4-f1e3eeafd7da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:12:11.970052 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.auth.core [None req-ad573d7a-5a6b-4f0c-98a4-f1e3eeafd7da None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81449) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:12:11.971710 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-ad573d7a-5a6b-4f0c-98a4-f1e3eeafd7da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:12:12.032959 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:12:12.044540 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 498/991] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 12:12:11 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 15 12:12:12.307430 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9b6dc6-6843-47ae-98dd-5f778075d4e6 None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:12:12.409698 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9b6dc6-6843-47ae-98dd-5f778075d4e6 None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:12:12.444206 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee347e4-0125-47c2-8e34-276a27cbc300 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:12:12.444999 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee347e4-0125-47c2-8e34-276a27cbc300 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:12:12.445284 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee347e4-0125-47c2-8e34-276a27cbc300 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:12:12.445558 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee347e4-0125-47c2-8e34-276a27cbc300 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:12:12.446142 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee347e4-0125-47c2-8e34-276a27cbc300 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:12:12.447495 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee347e4-0125-47c2-8e34-276a27cbc300 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:12:12.456578 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application [None req-6ee347e4-0125-47c2-8e34-276a27cbc300 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 15 12:12:12.456578 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:12:12.456578 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 15 12:12:12.456578 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 15 12:12:12.456578 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 15 12:12:12.456578 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 15 12:12:12.456578 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 15 12:12:12.456578 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 15 12:12:12.456578 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 15 12:12:12.456578 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 15 12:12:12.456578 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 15 12:12:12.456578 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 15 12:12:12.456578 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:12:12.456578 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:12:12.456578 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:12:12.456578 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:12:12.456578 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:12:12.457587 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:12:12.457587 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:12:12.457587 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:12:12.457587 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:12:12.457587 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:12:12.457587 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:12:12.457587 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 15 12:12:12.457587 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 15 12:12:12.457587 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 15 12:12:12.457587 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 15 12:12:12.457587 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:12:12.457587 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:12:12.457587 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:12:12.457587 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:12:12.457587 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:12:12.457587 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:12:12.458449 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:12:12.458449 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application with Lock( Sep 15 12:12:12.458449 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:12:12.458449 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:12:12.458449 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:12:12.458449 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:12:12.458449 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:12:12.458449 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:12:12.458449 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:12:12.458449 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:12:12.458449 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 15 12:12:12.458449 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 15 12:12:12.458449 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 15 12:12:12.458449 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 15 12:12:12.458449 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 494/992] 10.222.0.15 () {56 vars in 1111 bytes} [Thu Sep 15 12:12:12 2022] GET /identity/v3/services/compute => generated 88 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:12:12.465089 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad573d7a-5a6b-4f0c-98a4-f1e3eeafd7da None None] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:12:12.571516 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad573d7a-5a6b-4f0c-98a4-f1e3eeafd7da None None] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:12:12.606499 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513df076-cc2f-410b-a415-cb42809c852a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:12:12.607106 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513df076-cc2f-410b-a415-cb42809c852a None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:12:12.607380 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513df076-cc2f-410b-a415-cb42809c852a None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:12:12.607644 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513df076-cc2f-410b-a415-cb42809c852a None admin] PATH_INFO: `/v3/services` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:12:12.608261 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513df076-cc2f-410b-a415-cb42809c852a None admin] RBAC: Adding query filter params (name=compute) {{(pid=81449) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:12:12.608558 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513df076-cc2f-410b-a415-cb42809c852a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:12:12.609934 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513df076-cc2f-410b-a415-cb42809c852a None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:12:12.619995 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 499/993] 10.222.0.15 () {56 vars in 1112 bytes} [Thu Sep 15 12:12:12 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:12:12.620957 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee347e4-0125-47c2-8e34-276a27cbc300 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:12:12.660952 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440ec587-9b5a-4a89-a6ba-002a9c884333 None None] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:12:12.661358 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440ec587-9b5a-4a89-a6ba-002a9c884333 None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:12:12.661560 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440ec587-9b5a-4a89-a6ba-002a9c884333 None None] PATH_INFO: `/` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:12:12.662960 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 500/994] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 12:12:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 15 12:12:12.668234 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8dac8c-e61a-43ba-ac2b-d4ec5d1b1b2b None None] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:12:12.668510 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8dac8c-e61a-43ba-ac2b-d4ec5d1b1b2b None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:12:12.668815 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8dac8c-e61a-43ba-ac2b-d4ec5d1b1b2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:12:12.679842 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:12:12.690920 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.auth.core [None req-2d8dac8c-e61a-43ba-ac2b-d4ec5d1b1b2b None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81449) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:12:12.692100 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-2d8dac8c-e61a-43ba-ac2b-d4ec5d1b1b2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:12:12.729280 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee347e4-0125-47c2-8e34-276a27cbc300 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:12:12.746867 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:12:12.764508 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 501/995] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 12:12:12 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 15 12:12:12.766457 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d2465b-6447-40b3-9a68-67a188d86bf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:12:12.767322 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d2465b-6447-40b3-9a68-67a188d86bf9 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:12:12.767805 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d2465b-6447-40b3-9a68-67a188d86bf9 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:12:12.768304 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d2465b-6447-40b3-9a68-67a188d86bf9 None admin] PATH_INFO: `/v3/services` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:12:12.769179 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d2465b-6447-40b3-9a68-67a188d86bf9 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:12:12.769684 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d2465b-6447-40b3-9a68-67a188d86bf9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:12:12.771229 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d2465b-6447-40b3-9a68-67a188d86bf9 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:12:12.780906 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 495/996] 10.222.0.15 () {56 vars in 1112 bytes} [Thu Sep 15 12:12:12 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 12:12:12.782067 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8dac8c-e61a-43ba-ac2b-d4ec5d1b1b2b None None] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:12:12.880468 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8dac8c-e61a-43ba-ac2b-d4ec5d1b1b2b None None] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:12:12.914101 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e73ceed-4379-4eb8-9302-7337f0dc78bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:12:12.914769 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e73ceed-4379-4eb8-9302-7337f0dc78bd None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:12:12.915110 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e73ceed-4379-4eb8-9302-7337f0dc78bd None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:12:12.915382 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e73ceed-4379-4eb8-9302-7337f0dc78bd None admin] PATH_INFO: `/v3/endpoints` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:12:12.916075 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e73ceed-4379-4eb8-9302-7337f0dc78bd None admin] RBAC: Adding query filter params (interface=public, service_id=ebbeac6f7a444fd5b1fc9ae55f78812a, region_id=RegionOne) {{(pid=81449) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:12:12.916378 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e73ceed-4379-4eb8-9302-7337f0dc78bd None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:12:12.917751 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e73ceed-4379-4eb8-9302-7337f0dc78bd None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:12:12.922915 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 502/997] 10.222.0.15 () {56 vars in 1251 bytes} [Thu Sep 15 12:12:12 2022] GET /identity/v3/endpoints?service_id=ebbeac6f7a444fd5b1fc9ae55f78812a&interface=public®ion_id=RegionOne => generated 194 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:12:12.928596 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d2465b-6447-40b3-9a68-67a188d86bf9 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:12:13.034677 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e73ceed-4379-4eb8-9302-7337f0dc78bd None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:12:13.034877 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d2465b-6447-40b3-9a68-67a188d86bf9 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:12:13.057353 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:12:13.073532 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01287d4c-32dc-4fe8-9d9a-7783fcb90a51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:12:13.074143 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01287d4c-32dc-4fe8-9d9a-7783fcb90a51 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:12:13.074418 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01287d4c-32dc-4fe8-9d9a-7783fcb90a51 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:12:13.074686 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01287d4c-32dc-4fe8-9d9a-7783fcb90a51 None admin] PATH_INFO: `/v3/services` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:12:13.075282 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01287d4c-32dc-4fe8-9d9a-7783fcb90a51 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:12:13.076627 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01287d4c-32dc-4fe8-9d9a-7783fcb90a51 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:12:13.083503 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 496/998] 10.222.0.15 () {56 vars in 1087 bytes} [Thu Sep 15 12:12:12 2022] GET /identity/v3/services => generated 1727 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 12:12:13.088964 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:12:13.147436 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e73ceed-4379-4eb8-9302-7337f0dc78bd None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:12:13.193293 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e52c29d-721d-45a1-b945-47142c6b0d13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:12:13.193958 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e52c29d-721d-45a1-b945-47142c6b0d13 None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:12:13.194254 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e52c29d-721d-45a1-b945-47142c6b0d13 None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:12:13.194509 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e52c29d-721d-45a1-b945-47142c6b0d13 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:12:13.195075 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e52c29d-721d-45a1-b945-47142c6b0d13 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:12:13.199291 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e52c29d-721d-45a1-b945-47142c6b0d13 None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:12:13.209440 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application [None req-5e52c29d-721d-45a1-b945-47142c6b0d13 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 15 12:12:13.209440 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:12:13.209440 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 15 12:12:13.209440 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 15 12:12:13.209440 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 15 12:12:13.209440 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 15 12:12:13.209440 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 15 12:12:13.209440 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 15 12:12:13.209440 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 15 12:12:13.209440 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application Sep 15 12:12:13.209440 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 15 12:12:13.209440 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application Sep 15 12:12:13.209440 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:12:13.209440 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:12:13.209440 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:12:13.209440 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:12:13.209440 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:12:13.210545 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:12:13.210545 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:12:13.210545 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:12:13.210545 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:12:13.210545 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:12:13.210545 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:12:13.210545 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 15 12:12:13.210545 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 15 12:12:13.210545 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 15 12:12:13.210545 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 15 12:12:13.210545 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:12:13.210545 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:12:13.210545 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:12:13.210545 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:12:13.210545 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:12:13.210545 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:12:13.211519 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:12:13.211519 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application with Lock( Sep 15 12:12:13.211519 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:12:13.211519 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:12:13.211519 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:12:13.211519 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:12:13.211519 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:12:13.211519 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:12:13.211519 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:12:13.211519 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:12:13.211519 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 15 12:12:13.211519 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 15 12:12:13.211519 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Sep 15 12:12:13.211519 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application Sep 15 12:12:13.215915 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 503/999] 10.222.0.15 () {56 vars in 1126 bytes} [Thu Sep 15 12:12:13 2022] GET /identity/v3/services/object-store => generated 93 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 15 12:12:13.217209 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01287d4c-32dc-4fe8-9d9a-7783fcb90a51 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:12:13.328546 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01287d4c-32dc-4fe8-9d9a-7783fcb90a51 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:12:13.361724 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3aead3-1722-4fca-9fab-03876bae1700 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:12:13.362903 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3aead3-1722-4fca-9fab-03876bae1700 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:12:13.363181 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3aead3-1722-4fca-9fab-03876bae1700 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:12:13.363445 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea3aead3-1722-4fca-9fab-03876bae1700 None admin] PATH_INFO: `/v3/services` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:12:13.364062 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3aead3-1722-4fca-9fab-03876bae1700 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:12:13.364358 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3aead3-1722-4fca-9fab-03876bae1700 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:12:13.365720 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea3aead3-1722-4fca-9fab-03876bae1700 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:12:13.370666 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 497/1000] 10.222.0.15 () {56 vars in 1122 bytes} [Thu Sep 15 12:12:13 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:12:13.376116 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e52c29d-721d-45a1-b945-47142c6b0d13 None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:12:13.476870 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e52c29d-721d-45a1-b945-47142c6b0d13 None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:12:13.490482 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:12:13.512385 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34eb1008-563d-4a3f-951e-afb75c51e9c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:12:13.513017 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34eb1008-563d-4a3f-951e-afb75c51e9c3 None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:12:13.513294 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34eb1008-563d-4a3f-951e-afb75c51e9c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:12:13.513561 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34eb1008-563d-4a3f-951e-afb75c51e9c3 None admin] PATH_INFO: `/v3/services` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:12:13.514179 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34eb1008-563d-4a3f-951e-afb75c51e9c3 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81449) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:12:13.514475 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34eb1008-563d-4a3f-951e-afb75c51e9c3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:12:13.515823 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34eb1008-563d-4a3f-951e-afb75c51e9c3 None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:12:13.521365 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 504/1001] 10.222.0.15 () {56 vars in 1122 bytes} [Thu Sep 15 12:12:13 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:12:13.526739 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3aead3-1722-4fca-9fab-03876bae1700 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:12:13.578568 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:12:13.631179 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea3aead3-1722-4fca-9fab-03876bae1700 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:12:13.663911 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b744b8-6c92-41e7-8fca-380217d85204 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:12:13.664521 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b744b8-6c92-41e7-8fca-380217d85204 None admin] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:12:13.664944 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b744b8-6c92-41e7-8fca-380217d85204 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:12:13.665262 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b744b8-6c92-41e7-8fca-380217d85204 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:12:13.665862 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b744b8-6c92-41e7-8fca-380217d85204 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:12:13.667202 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b744b8-6c92-41e7-8fca-380217d85204 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:12:13.684647 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 498/1002] 10.222.0.15 () {60 vars in 1144 bytes} [Thu Sep 15 12:12:13 2022] POST /identity/v3/endpoints => generated 325 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 12:12:13.989421 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47854ff2-f465-45b5-8a03-82b797cf3884 None None] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:12:13.989764 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47854ff2-f465-45b5-8a03-82b797cf3884 None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:12:13.989981 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47854ff2-f465-45b5-8a03-82b797cf3884 None None] PATH_INFO: `/` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:12:13.991456 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 505/1003] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 12:12:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:12:13.996714 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d7837d-2a6d-46a5-b82d-4b6af43855bc None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:12:13.997042 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d7837d-2a6d-46a5-b82d-4b6af43855bc None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:12:13.997320 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d7837d-2a6d-46a5-b82d-4b6af43855bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:12:14.015679 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-d6d7837d-2a6d-46a5-b82d-4b6af43855bc None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:12:14.016886 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-d6d7837d-2a6d-46a5-b82d-4b6af43855bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:12:14.087065 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 499/1004] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 12:12:13 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 15 12:12:14.353021 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47854ff2-f465-45b5-8a03-82b797cf3884 None None] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:12:14.457831 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47854ff2-f465-45b5-8a03-82b797cf3884 None None] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:12:14.491292 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92549821-b537-4bf4-87e4-c183273aea9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:12:14.492004 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92549821-b537-4bf4-87e4-c183273aea9e None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:12:14.492191 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92549821-b537-4bf4-87e4-c183273aea9e None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:12:14.492519 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92549821-b537-4bf4-87e4-c183273aea9e None admin] PATH_INFO: `/v3/services/image` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:12:14.493127 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92549821-b537-4bf4-87e4-c183273aea9e None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:12:14.494601 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92549821-b537-4bf4-87e4-c183273aea9e None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:12:14.503451 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application [None req-92549821-b537-4bf4-87e4-c183273aea9e None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 15 12:12:14.503451 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:12:14.503451 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 15 12:12:14.503451 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 15 12:12:14.503451 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 15 12:12:14.503451 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 15 12:12:14.503451 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 15 12:12:14.503451 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 15 12:12:14.503451 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 15 12:12:14.503451 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application Sep 15 12:12:14.503451 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 15 12:12:14.503451 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application Sep 15 12:12:14.503451 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:12:14.503451 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:12:14.503451 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:12:14.503451 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:12:14.503451 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:12:14.504423 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:12:14.504423 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:12:14.504423 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:12:14.504423 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:12:14.504423 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:12:14.504423 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:12:14.504423 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 15 12:12:14.504423 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 15 12:12:14.504423 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 15 12:12:14.504423 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 15 12:12:14.504423 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:12:14.504423 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:12:14.504423 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:12:14.504423 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:12:14.504423 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:12:14.504423 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:12:14.505700 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:12:14.505700 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application with Lock( Sep 15 12:12:14.505700 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:12:14.505700 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:12:14.505700 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:12:14.505700 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:12:14.505700 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:12:14.505700 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:12:14.505700 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:12:14.505700 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:12:14.505700 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 15 12:12:14.505700 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 15 12:12:14.505700 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 15 12:12:14.505700 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application Sep 15 12:12:14.509082 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 506/1005] 10.222.0.15 () {56 vars in 1105 bytes} [Thu Sep 15 12:12:14 2022] GET /identity/v3/services/image => generated 86 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:12:14.510409 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d7837d-2a6d-46a5-b82d-4b6af43855bc None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:12:14.546222 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:12:14.617529 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d7837d-2a6d-46a5-b82d-4b6af43855bc None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:12:14.651587 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7500942-462c-42ba-a460-e4672c290c42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:12:14.652196 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7500942-462c-42ba-a460-e4672c290c42 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:12:14.652472 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7500942-462c-42ba-a460-e4672c290c42 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:12:14.652740 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7500942-462c-42ba-a460-e4672c290c42 None admin] PATH_INFO: `/v3/services` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:12:14.653386 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7500942-462c-42ba-a460-e4672c290c42 None admin] RBAC: Adding query filter params (name=image) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:12:14.653685 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7500942-462c-42ba-a460-e4672c290c42 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:12:14.655068 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7500942-462c-42ba-a460-e4672c290c42 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:12:14.660589 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 500/1006] 10.222.0.15 () {56 vars in 1108 bytes} [Thu Sep 15 12:12:14 2022] GET /identity/v3/services?name=image => generated 122 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:12:14.666113 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92549821-b537-4bf4-87e4-c183273aea9e None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:12:14.710591 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:12:14.769361 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92549821-b537-4bf4-87e4-c183273aea9e None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:12:14.803392 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c1a22c-c766-4966-8d78-f1ade732a8e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:12:14.803992 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c1a22c-c766-4966-8d78-f1ade732a8e7 None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:12:14.804266 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c1a22c-c766-4966-8d78-f1ade732a8e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:12:14.804528 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c1a22c-c766-4966-8d78-f1ade732a8e7 None admin] PATH_INFO: `/v3/services` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:12:14.805166 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c1a22c-c766-4966-8d78-f1ade732a8e7 None admin] RBAC: Adding query filter params (type=image) {{(pid=81449) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:12:14.805462 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c1a22c-c766-4966-8d78-f1ade732a8e7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:12:14.806810 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c1a22c-c766-4966-8d78-f1ade732a8e7 None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:12:14.811451 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 507/1007] 10.222.0.15 () {56 vars in 1108 bytes} [Thu Sep 15 12:12:14 2022] GET /identity/v3/services?type=image => generated 122 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:12:15.264853 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764fd8a3-5898-4f2e-92ca-a77ab24bada9 None None] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:12:15.265196 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764fd8a3-5898-4f2e-92ca-a77ab24bada9 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:12:15.265370 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764fd8a3-5898-4f2e-92ca-a77ab24bada9 None None] PATH_INFO: `/` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:12:15.266843 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 501/1008] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 12:12:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:12:15.272544 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e324f2f-2c75-4871-aa9f-3ec12ecc3df9 None None] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:12:15.272860 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e324f2f-2c75-4871-aa9f-3ec12ecc3df9 None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:12:15.273136 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e324f2f-2c75-4871-aa9f-3ec12ecc3df9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:12:15.292212 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.auth.core [None req-0e324f2f-2c75-4871-aa9f-3ec12ecc3df9 None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81449) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:12:15.293446 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-0e324f2f-2c75-4871-aa9f-3ec12ecc3df9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:12:15.360026 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 508/1009] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 12:12:15 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 15 12:12:15.626979 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764fd8a3-5898-4f2e-92ca-a77ab24bada9 None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:12:15.725872 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:12:15.731840 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764fd8a3-5898-4f2e-92ca-a77ab24bada9 None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:12:15.766897 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a773b249-0ea6-4e52-bbfb-fac728738ca4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:12:15.767560 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a773b249-0ea6-4e52-bbfb-fac728738ca4 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:12:15.767831 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a773b249-0ea6-4e52-bbfb-fac728738ca4 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:12:15.768098 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a773b249-0ea6-4e52-bbfb-fac728738ca4 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:12:15.769260 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a773b249-0ea6-4e52-bbfb-fac728738ca4 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:12:15.770660 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a773b249-0ea6-4e52-bbfb-fac728738ca4 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:12:15.780253 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application [None req-a773b249-0ea6-4e52-bbfb-fac728738ca4 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Sep 15 12:12:15.780253 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:12:15.780253 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 15 12:12:15.780253 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 15 12:12:15.780253 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 15 12:12:15.780253 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 15 12:12:15.780253 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 15 12:12:15.780253 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 15 12:12:15.780253 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 15 12:12:15.780253 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 15 12:12:15.780253 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 15 12:12:15.780253 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 15 12:12:15.780253 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:12:15.780253 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:12:15.780253 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:12:15.780253 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:12:15.780253 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:12:15.781619 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:12:15.781619 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:12:15.781619 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:12:15.781619 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:12:15.781619 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:12:15.781619 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:12:15.781619 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 15 12:12:15.781619 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 15 12:12:15.781619 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 15 12:12:15.781619 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 15 12:12:15.781619 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:12:15.781619 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:12:15.781619 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:12:15.781619 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:12:15.781619 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:12:15.781619 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:12:15.782497 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:12:15.782497 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application with Lock( Sep 15 12:12:15.782497 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:12:15.782497 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:12:15.782497 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:12:15.782497 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:12:15.782497 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:12:15.782497 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:12:15.782497 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:12:15.782497 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:12:15.782497 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 15 12:12:15.782497 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 15 12:12:15.782497 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Sep 15 12:12:15.782497 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 15 12:12:15.785107 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 502/1010] 10.222.0.15 () {56 vars in 1111 bytes} [Thu Sep 15 12:12:15 2022] GET /identity/v3/services/compute => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:12:15.787415 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e324f2f-2c75-4871-aa9f-3ec12ecc3df9 None None] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:12:15.821636 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:12:15.874126 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66cead7-4b92-4e9d-88e2-b657a741b0ce None None] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:12:15.874500 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66cead7-4b92-4e9d-88e2-b657a741b0ce None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:12:15.874680 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66cead7-4b92-4e9d-88e2-b657a741b0ce None None] PATH_INFO: `/` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:12:15.876015 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 503/1011] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 12:12:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 15 12:12:15.881636 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b1b20e-f8bf-43ac-a5a4-f8ed9eb428a0 None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:12:15.881855 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b1b20e-f8bf-43ac-a5a4-f8ed9eb428a0 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:12:15.882125 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b1b20e-f8bf-43ac-a5a4-f8ed9eb428a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:12:15.894041 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e324f2f-2c75-4871-aa9f-3ec12ecc3df9 None None] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:12:15.901012 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-c8b1b20e-f8bf-43ac-a5a4-f8ed9eb428a0 None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:12:15.902191 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-c8b1b20e-f8bf-43ac-a5a4-f8ed9eb428a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:12:15.927676 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a90b501-d6dd-4ceb-bd9f-400ff40d93a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:12:15.928284 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a90b501-d6dd-4ceb-bd9f-400ff40d93a9 None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:12:15.928559 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a90b501-d6dd-4ceb-bd9f-400ff40d93a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:12:15.928850 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a90b501-d6dd-4ceb-bd9f-400ff40d93a9 None admin] PATH_INFO: `/v3/services` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:12:15.929466 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a90b501-d6dd-4ceb-bd9f-400ff40d93a9 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81449) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:12:15.929751 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a90b501-d6dd-4ceb-bd9f-400ff40d93a9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:12:15.931101 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a90b501-d6dd-4ceb-bd9f-400ff40d93a9 None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:12:15.935908 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 509/1012] 10.222.0.15 () {56 vars in 1112 bytes} [Thu Sep 15 12:12:15 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 12:12:15.941704 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a90b501-d6dd-4ceb-bd9f-400ff40d93a9 None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:12:15.979765 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 504/1013] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 12:12:15 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 15 12:12:16.046052 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:12:16.048940 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a90b501-d6dd-4ceb-bd9f-400ff40d93a9 None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:12:16.084170 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a232d7d5-eb33-45db-a13c-81e1be8f6d35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:12:16.084795 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a232d7d5-eb33-45db-a13c-81e1be8f6d35 None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:12:16.085084 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a232d7d5-eb33-45db-a13c-81e1be8f6d35 None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:12:16.085401 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a232d7d5-eb33-45db-a13c-81e1be8f6d35 None admin] PATH_INFO: `/v3/services` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:12:16.086020 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a232d7d5-eb33-45db-a13c-81e1be8f6d35 None admin] RBAC: Adding query filter params (type=compute) {{(pid=81449) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:12:16.086318 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a232d7d5-eb33-45db-a13c-81e1be8f6d35 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:12:16.087665 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a232d7d5-eb33-45db-a13c-81e1be8f6d35 None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:12:16.092835 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 510/1014] 10.222.0.15 () {56 vars in 1112 bytes} [Thu Sep 15 12:12:15 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 12:12:16.098215 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b1b20e-f8bf-43ac-a5a4-f8ed9eb428a0 None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:12:16.199859 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:12:16.210096 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b1b20e-f8bf-43ac-a5a4-f8ed9eb428a0 None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:12:16.249533 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32007ee6-7880-40ee-8830-41f0eee186ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:12:16.250267 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32007ee6-7880-40ee-8830-41f0eee186ef None admin] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:12:16.250540 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32007ee6-7880-40ee-8830-41f0eee186ef None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:12:16.250800 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32007ee6-7880-40ee-8830-41f0eee186ef None admin] PATH_INFO: `/v3/endpoints` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:12:16.251458 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32007ee6-7880-40ee-8830-41f0eee186ef None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:12:16.253384 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32007ee6-7880-40ee-8830-41f0eee186ef None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:12:16.253649 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a232d7d5-eb33-45db-a13c-81e1be8f6d35 None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:12:16.276016 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 505/1015] 10.222.0.15 () {60 vars in 1144 bytes} [Thu Sep 15 12:12:16 2022] POST /identity/v3/endpoints => generated 334 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 12:12:16.366356 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a232d7d5-eb33-45db-a13c-81e1be8f6d35 None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:12:16.409596 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321bcfe6-460f-4138-8ea9-3b2b26e6acfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:12:16.410246 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321bcfe6-460f-4138-8ea9-3b2b26e6acfc None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:12:16.410524 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321bcfe6-460f-4138-8ea9-3b2b26e6acfc None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:12:16.410874 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321bcfe6-460f-4138-8ea9-3b2b26e6acfc None admin] PATH_INFO: `/v3/domains/default` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:12:16.412022 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321bcfe6-460f-4138-8ea9-3b2b26e6acfc None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:12:16.413513 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321bcfe6-460f-4138-8ea9-3b2b26e6acfc None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:12:16.415363 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 511/1016] 10.222.0.15 () {56 vars in 1108 bytes} [Thu Sep 15 12:12:16 2022] GET /identity/v3/domains/default => generated 199 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 12:12:16.421018 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32007ee6-7880-40ee-8830-41f0eee186ef None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:12:16.531480 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32007ee6-7880-40ee-8830-41f0eee186ef None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:12:16.569629 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc74e89c-98e0-4524-95b2-bd80f1c65305 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:12:16.570244 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc74e89c-98e0-4524-95b2-bd80f1c65305 None admin] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:12:16.570528 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc74e89c-98e0-4524-95b2-bd80f1c65305 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:12:16.570798 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc74e89c-98e0-4524-95b2-bd80f1c65305 None admin] PATH_INFO: `/v3/projects` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:12:16.571452 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc74e89c-98e0-4524-95b2-bd80f1c65305 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:12:16.572901 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc74e89c-98e0-4524-95b2-bd80f1c65305 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:12:16.596248 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:12:16.605217 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 506/1017] 10.222.0.15 () {60 vars in 1141 bytes} [Thu Sep 15 12:12:16 2022] POST /identity/v3/projects => generated 311 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 12:12:16.974654 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d40b47-b326-45e9-8ea9-43477c4c45b5 None None] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:12:16.975166 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d40b47-b326-45e9-8ea9-43477c4c45b5 None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:12:16.975166 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d40b47-b326-45e9-8ea9-43477c4c45b5 None None] PATH_INFO: `/` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:12:16.976955 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 512/1018] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 12:12:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:12:16.982408 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893600f8-eeb3-4a5a-b187-92038a716d42 None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:12:16.982790 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893600f8-eeb3-4a5a-b187-92038a716d42 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:12:16.982982 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893600f8-eeb3-4a5a-b187-92038a716d42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:12:17.001463 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-893600f8-eeb3-4a5a-b187-92038a716d42 None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:12:17.003245 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-893600f8-eeb3-4a5a-b187-92038a716d42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:12:17.070784 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 507/1019] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 12:12:16 2022] POST /identity/v3/auth/tokens => generated 1873 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 15 12:12:17.345647 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d40b47-b326-45e9-8ea9-43477c4c45b5 None None] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:12:17.374726 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:12:17.448739 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d40b47-b326-45e9-8ea9-43477c4c45b5 None None] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:12:17.482519 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf855eb-344f-4791-a6e8-d91e0b69b4ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:12:17.483121 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf855eb-344f-4791-a6e8-d91e0b69b4ce None admin] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:12:17.483394 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf855eb-344f-4791-a6e8-d91e0b69b4ce None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:12:17.483662 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf855eb-344f-4791-a6e8-d91e0b69b4ce None admin] PATH_INFO: `/v3/services` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:12:17.484307 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf855eb-344f-4791-a6e8-d91e0b69b4ce None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:12:17.485696 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf855eb-344f-4791-a6e8-d91e0b69b4ce None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:12:17.502718 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 513/1020] 10.222.0.15 () {60 vars in 1141 bytes} [Thu Sep 15 12:12:17 2022] POST /identity/v3/services => generated 242 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 12:12:18.589252 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41273bf7-25bd-4911-a1da-c3d397adbaba None None] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:12:18.589776 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41273bf7-25bd-4911-a1da-c3d397adbaba None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:12:18.589776 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41273bf7-25bd-4911-a1da-c3d397adbaba None None] PATH_INFO: `/` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:12:18.591399 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 508/1021] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 12:12:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:12:18.596315 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0f1fa4-b226-4ac8-aa47-277f8f87f2fa None None] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:12:18.596707 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0f1fa4-b226-4ac8-aa47-277f8f87f2fa None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:12:18.596889 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0f1fa4-b226-4ac8-aa47-277f8f87f2fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:12:18.616593 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.auth.core [None req-5e0f1fa4-b226-4ac8-aa47-277f8f87f2fa None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81449) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:12:18.618078 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-5e0f1fa4-b226-4ac8-aa47-277f8f87f2fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:12:18.694207 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 514/1022] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 12:12:18 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 15 12:12:18.815775 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460a332c-8a90-4881-a4fb-29881566e7e6 None None] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:12:18.816063 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460a332c-8a90-4881-a4fb-29881566e7e6 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:12:18.816545 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460a332c-8a90-4881-a4fb-29881566e7e6 None None] PATH_INFO: `/` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:12:18.818058 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 509/1023] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 12:12:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:12:18.823036 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28aeac46-866a-4d82-967b-06a097d8a2cf None None] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:12:18.823332 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28aeac46-866a-4d82-967b-06a097d8a2cf None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:12:18.823606 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28aeac46-866a-4d82-967b-06a097d8a2cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:12:18.842138 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.auth.core [None req-28aeac46-866a-4d82-967b-06a097d8a2cf None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81449) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:12:18.843248 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-28aeac46-866a-4d82-967b-06a097d8a2cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:12:18.853447 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:12:18.915075 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 515/1024] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 12:12:18 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 15 12:12:18.964075 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460a332c-8a90-4881-a4fb-29881566e7e6 None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:12:19.064584 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460a332c-8a90-4881-a4fb-29881566e7e6 None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:12:19.093034 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:12:19.099774 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c255ba-66d4-470e-a879-64246f325502 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:12:19.100413 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c255ba-66d4-470e-a879-64246f325502 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:12:19.100691 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c255ba-66d4-470e-a879-64246f325502 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:12:19.100989 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c255ba-66d4-470e-a879-64246f325502 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:12:19.111222 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c255ba-66d4-470e-a879-64246f325502 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 12:12:19.116570 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application [None req-a3c255ba-66d4-470e-a879-64246f325502 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 15 12:12:19.116570 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:12:19.116570 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:12:19.116570 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:12:19.116570 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:12:19.116570 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:12:19.116570 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:12:19.116570 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:12:19.116570 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:12:19.116570 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:12:19.116570 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:12:19.116570 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:12:19.116570 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 12:12:19.116570 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 12:12:19.116570 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 12:12:19.116570 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 12:12:19.117537 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 12:12:19.117537 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 12:12:19.117537 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 12:12:19.117537 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 12:12:19.117537 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:12:19.117537 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:12:19.117537 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:12:19.117537 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:12:19.117537 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:12:19.117537 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:12:19.117537 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:12:19.117537 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application with Lock( Sep 15 12:12:19.117537 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:12:19.117537 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:12:19.117537 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:12:19.117537 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:12:19.118386 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:12:19.118386 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:12:19.118386 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:12:19.118386 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:12:19.118386 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 12:12:19.118386 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 12:12:19.118386 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 12:12:19.118386 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 12:12:19.118386 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 12:12:19.118386 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 12:12:19.118386 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 15 12:12:19.118386 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 15 12:12:19.121120 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 510/1025] 10.222.0.15 () {56 vars in 1120 bytes} [Thu Sep 15 12:12:18 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:12:19.123666 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28aeac46-866a-4d82-967b-06a097d8a2cf None None] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:12:19.180680 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c255ba-66d4-470e-a879-64246f325502 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:12:19.224466 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28aeac46-866a-4d82-967b-06a097d8a2cf None None] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:12:19.263430 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd33ba1-3491-4774-b3ba-b73e5d0c4dfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:12:19.264031 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd33ba1-3491-4774-b3ba-b73e5d0c4dfa None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:12:19.264312 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd33ba1-3491-4774-b3ba-b73e5d0c4dfa None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:12:19.264579 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd33ba1-3491-4774-b3ba-b73e5d0c4dfa None admin] PATH_INFO: `/v3/roles` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:12:19.265271 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd33ba1-3491-4774-b3ba-b73e5d0c4dfa None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81449) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:12:19.265576 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd33ba1-3491-4774-b3ba-b73e5d0c4dfa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:12:19.266937 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd33ba1-3491-4774-b3ba-b73e5d0c4dfa None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:12:19.280125 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 516/1026] 10.222.0.15 () {56 vars in 1115 bytes} [Thu Sep 15 12:12:19 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:12:19.281640 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd33ba1-3491-4774-b3ba-b73e5d0c4dfa None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:12:19.290743 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:12:19.293226 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c255ba-66d4-470e-a879-64246f325502 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:12:19.333524 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef94ccf-d93b-47e0-a515-5f327699b196 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:12:19.334172 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef94ccf-d93b-47e0-a515-5f327699b196 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:12:19.334450 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef94ccf-d93b-47e0-a515-5f327699b196 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:12:19.334718 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef94ccf-d93b-47e0-a515-5f327699b196 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:12:19.335802 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef94ccf-d93b-47e0-a515-5f327699b196 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:12:19.337704 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef94ccf-d93b-47e0-a515-5f327699b196 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:12:19.339423 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 511/1027] 10.222.0.15 () {56 vars in 1108 bytes} [Thu Sep 15 12:12:19 2022] GET /identity/v3/domains/default => generated 199 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 12:12:19.344811 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef94ccf-d93b-47e0-a515-5f327699b196 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:12:19.391549 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd33ba1-3491-4774-b3ba-b73e5d0c4dfa None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:12:19.407427 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:12:19.428086 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a0c954-1ee0-41f7-9d8a-6eeaa1105e65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:12:19.429114 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a0c954-1ee0-41f7-9d8a-6eeaa1105e65 None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:12:19.429405 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a0c954-1ee0-41f7-9d8a-6eeaa1105e65 None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:12:19.429680 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a0c954-1ee0-41f7-9d8a-6eeaa1105e65 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:12:19.441008 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a0c954-1ee0-41f7-9d8a-6eeaa1105e65 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:12:19.442457 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a0c954-1ee0-41f7-9d8a-6eeaa1105e65 None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:12:19.453577 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef94ccf-d93b-47e0-a515-5f327699b196 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:12:19.457779 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application [None req-c1a0c954-1ee0-41f7-9d8a-6eeaa1105e65 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 12:12:19.457779 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:12:19.457779 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:12:19.457779 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:12:19.457779 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:12:19.457779 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:12:19.457779 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:12:19.457779 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:12:19.457779 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:12:19.457779 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:12:19.457779 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:12:19.457779 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:12:19.457779 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 12:12:19.457779 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 12:12:19.457779 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 12:12:19.457779 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 12:12:19.458709 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:12:19.458709 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:12:19.458709 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:12:19.458709 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:12:19.458709 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:12:19.458709 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:12:19.458709 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:12:19.458709 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application with Lock( Sep 15 12:12:19.458709 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:12:19.458709 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:12:19.458709 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:12:19.458709 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:12:19.458709 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:12:19.458709 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:12:19.458709 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:12:19.458709 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:12:19.458709 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 12:12:19.459618 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 12:12:19.459618 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 12:12:19.459618 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application Sep 15 12:12:19.459618 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 517/1028] 10.222.0.15 () {56 vars in 1108 bytes} [Thu Sep 15 12:12:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 15 12:12:19.464540 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a0c954-1ee0-41f7-9d8a-6eeaa1105e65 None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:12:19.491603 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8208a9b1-f211-419c-bcbd-67ccde186fad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:12:19.492259 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8208a9b1-f211-419c-bcbd-67ccde186fad None admin] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:12:19.492499 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8208a9b1-f211-419c-bcbd-67ccde186fad None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:12:19.492785 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8208a9b1-f211-419c-bcbd-67ccde186fad None admin] PATH_INFO: `/v3/users` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:12:19.493432 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8208a9b1-f211-419c-bcbd-67ccde186fad None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:12:19.494850 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8208a9b1-f211-419c-bcbd-67ccde186fad None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:12:19.519683 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 512/1029] 10.222.0.15 () {60 vars in 1132 bytes} [Thu Sep 15 12:12:19 2022] POST /identity/v3/users => generated 285 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 15 12:12:19.526830 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:12:19.579318 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a0c954-1ee0-41f7-9d8a-6eeaa1105e65 None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:12:19.623264 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7002ced6-b50a-4392-b2b8-7978570fb146 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:12:19.623906 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7002ced6-b50a-4392-b2b8-7978570fb146 None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:12:19.624122 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7002ced6-b50a-4392-b2b8-7978570fb146 None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:12:19.624438 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7002ced6-b50a-4392-b2b8-7978570fb146 None admin] PATH_INFO: `/v3/domains` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:12:19.625125 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7002ced6-b50a-4392-b2b8-7978570fb146 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81449) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:12:19.625481 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7002ced6-b50a-4392-b2b8-7978570fb146 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:12:19.626876 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7002ced6-b50a-4392-b2b8-7978570fb146 None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:12:19.644950 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 518/1030] 10.222.0.15 () {56 vars in 1109 bytes} [Thu Sep 15 12:12:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 12:12:19.647570 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8208a9b1-f211-419c-bcbd-67ccde186fad None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:12:19.698616 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123fb47e-8c46-43e3-9d6c-5c2aaf4c7f08 None None] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:12:19.698936 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123fb47e-8c46-43e3-9d6c-5c2aaf4c7f08 None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:12:19.699180 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123fb47e-8c46-43e3-9d6c-5c2aaf4c7f08 None None] PATH_INFO: `/` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:12:19.700542 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 519/1031] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 12:12:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 15 12:12:19.706356 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d582718-d764-4c13-93d7-84d8c5eafca1 None None] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:12:19.706642 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d582718-d764-4c13-93d7-84d8c5eafca1 None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:12:19.706917 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d582718-d764-4c13-93d7-84d8c5eafca1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:12:19.726122 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.auth.core [None req-6d582718-d764-4c13-93d7-84d8c5eafca1 None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81449) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:12:19.727316 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-6d582718-d764-4c13-93d7-84d8c5eafca1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:12:19.770137 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8208a9b1-f211-419c-bcbd-67ccde186fad None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:12:19.785491 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:12:19.809153 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35cd3a78-2985-458e-b7ee-158632be87ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:12:19.809793 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cd3a78-2985-458e-b7ee-158632be87ce None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:12:19.812946 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cd3a78-2985-458e-b7ee-158632be87ce None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:12:19.812946 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cd3a78-2985-458e-b7ee-158632be87ce None admin] PATH_INFO: `/v3/users/nova` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:12:19.813148 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 520/1032] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 12:12:19 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 15 12:12:19.819305 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35cd3a78-2985-458e-b7ee-158632be87ce None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:12:19.820769 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35cd3a78-2985-458e-b7ee-158632be87ce None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:12:19.831738 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application [None req-35cd3a78-2985-458e-b7ee-158632be87ce None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 15 12:12:19.831738 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:12:19.831738 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:12:19.831738 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:12:19.831738 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:12:19.831738 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:12:19.831738 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:12:19.831738 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:12:19.831738 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:12:19.831738 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:12:19.831738 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:12:19.831738 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:12:19.831738 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 15 12:12:19.831738 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 15 12:12:19.831738 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 15 12:12:19.831738 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 15 12:12:19.832702 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:12:19.832702 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:12:19.832702 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 15 12:12:19.832702 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 12:12:19.832702 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 15 12:12:19.832702 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 12:12:19.832702 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:12:19.832702 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:12:19.832702 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:12:19.832702 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:12:19.832702 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:12:19.832702 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application with Lock( Sep 15 12:12:19.832702 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:12:19.832702 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:12:19.832702 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:12:19.832702 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:12:19.832702 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:12:19.832702 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:12:19.833921 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:12:19.833921 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:12:19.833921 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 15 12:12:19.833921 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 15 12:12:19.833921 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 15 12:12:19.833921 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 15 12:12:19.833921 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 15 12:12:19.833921 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 15 12:12:19.833921 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 15 12:12:19.833921 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 15 12:12:19.833921 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 513/1033] 10.222.0.15 () {56 vars in 1093 bytes} [Thu Sep 15 12:12:19 2022] GET /identity/v3/users/nova => generated 82 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:12:19.838758 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d582718-d764-4c13-93d7-84d8c5eafca1 None None] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:12:19.889043 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:12:19.941171 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d582718-d764-4c13-93d7-84d8c5eafca1 None None] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:12:19.982954 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e55043-2572-410b-84ae-855298dc60ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:12:19.983654 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e55043-2572-410b-84ae-855298dc60ff None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:12:19.984004 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e55043-2572-410b-84ae-855298dc60ff None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:12:19.984344 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e55043-2572-410b-84ae-855298dc60ff None admin] PATH_INFO: `/v3/users` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:12:19.985196 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e55043-2572-410b-84ae-855298dc60ff None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81449) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:12:19.985503 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e55043-2572-410b-84ae-855298dc60ff None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:12:19.987457 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e55043-2572-410b-84ae-855298dc60ff None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:12:20.005401 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 521/1034] 10.222.0.15 () {56 vars in 1133 bytes} [Thu Sep 15 12:12:19 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:12:20.011147 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35cd3a78-2985-458e-b7ee-158632be87ce None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:12:20.107170 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e55043-2572-410b-84ae-855298dc60ff None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:12:20.110344 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35cd3a78-2985-458e-b7ee-158632be87ce None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:12:20.144415 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f114e7-1a9c-40df-92d2-f361521269e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:12:20.144969 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f114e7-1a9c-40df-92d2-f361521269e1 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:12:20.145247 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f114e7-1a9c-40df-92d2-f361521269e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:12:20.145513 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f114e7-1a9c-40df-92d2-f361521269e1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:12:20.191048 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:12:20.212612 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e55043-2572-410b-84ae-855298dc60ff None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:12:20.221961 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f114e7-1a9c-40df-92d2-f361521269e1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:12:20.224585 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f114e7-1a9c-40df-92d2-f361521269e1 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:12:20.228406 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:12:20.262417 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac33ab57-8c77-4a88-b10f-8de8ada01044 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:12:20.263059 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac33ab57-8c77-4a88-b10f-8de8ada01044 None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:12:20.263338 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac33ab57-8c77-4a88-b10f-8de8ada01044 None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:12:20.263604 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac33ab57-8c77-4a88-b10f-8de8ada01044 None admin] PATH_INFO: `/v3/services/image` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:12:20.264187 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac33ab57-8c77-4a88-b10f-8de8ada01044 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:12:20.265599 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac33ab57-8c77-4a88-b10f-8de8ada01044 None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:12:20.274700 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application [None req-ac33ab57-8c77-4a88-b10f-8de8ada01044 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 15 12:12:20.274700 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:12:20.274700 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 15 12:12:20.274700 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 15 12:12:20.274700 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 15 12:12:20.274700 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 15 12:12:20.274700 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 15 12:12:20.274700 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 15 12:12:20.274700 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 15 12:12:20.274700 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application Sep 15 12:12:20.274700 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 15 12:12:20.274700 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application Sep 15 12:12:20.274700 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:12:20.274700 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:12:20.274700 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:12:20.274700 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:12:20.274700 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:12:20.276289 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:12:20.276289 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:12:20.276289 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:12:20.276289 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:12:20.276289 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:12:20.276289 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:12:20.276289 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 15 12:12:20.276289 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 15 12:12:20.276289 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 15 12:12:20.276289 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 15 12:12:20.276289 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:12:20.276289 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:12:20.276289 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:12:20.276289 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:12:20.276289 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:12:20.276289 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:12:20.277691 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:12:20.277691 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application with Lock( Sep 15 12:12:20.277691 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:12:20.277691 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:12:20.277691 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:12:20.277691 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:12:20.277691 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:12:20.277691 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:12:20.277691 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:12:20.277691 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:12:20.277691 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 15 12:12:20.277691 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 15 12:12:20.277691 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 15 12:12:20.277691 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application Sep 15 12:12:20.277691 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 522/1035] 10.222.0.15 () {56 vars in 1105 bytes} [Thu Sep 15 12:12:20 2022] GET /identity/v3/services/image => generated 86 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 15 12:12:20.281547 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac33ab57-8c77-4a88-b10f-8de8ada01044 None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:12:20.337712 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 514/1036] 10.222.0.15 () {58 vars in 1282 bytes} [Thu Sep 15 12:12:20 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 15 12:12:20.343335 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f114e7-1a9c-40df-92d2-f361521269e1 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:12:20.384148 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac33ab57-8c77-4a88-b10f-8de8ada01044 None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:12:20.420932 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af511f8-6836-450f-8122-57da42cf195f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:12:20.421460 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af511f8-6836-450f-8122-57da42cf195f None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:12:20.421731 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af511f8-6836-450f-8122-57da42cf195f None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:12:20.421998 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af511f8-6836-450f-8122-57da42cf195f None admin] PATH_INFO: `/v3/services` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:12:20.422612 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af511f8-6836-450f-8122-57da42cf195f None admin] RBAC: Adding query filter params (name=image) {{(pid=81449) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:12:20.422908 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af511f8-6836-450f-8122-57da42cf195f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:12:20.424263 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af511f8-6836-450f-8122-57da42cf195f None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:12:20.424374 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:12:20.429444 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 523/1037] 10.222.0.15 () {56 vars in 1108 bytes} [Thu Sep 15 12:12:20 2022] GET /identity/v3/services?name=image => generated 122 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 12:12:20.434702 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af511f8-6836-450f-8122-57da42cf195f None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:12:20.453939 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f114e7-1a9c-40df-92d2-f361521269e1 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:12:20.474363 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:12:20.493828 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a285b92-f8fa-4aa5-94b2-783d6c5a5988 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:12:20.494461 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a285b92-f8fa-4aa5-94b2-783d6c5a5988 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:12:20.494736 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a285b92-f8fa-4aa5-94b2-783d6c5a5988 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:12:20.494995 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a285b92-f8fa-4aa5-94b2-783d6c5a5988 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:12:20.505199 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a285b92-f8fa-4aa5-94b2-783d6c5a5988 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:12:20.506931 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a285b92-f8fa-4aa5-94b2-783d6c5a5988 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:12:20.522480 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application [None req-0a285b92-f8fa-4aa5-94b2-783d6c5a5988 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 12:12:20.522480 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:12:20.522480 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:12:20.522480 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:12:20.522480 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:12:20.522480 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:12:20.522480 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:12:20.522480 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:12:20.522480 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:12:20.522480 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:12:20.522480 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:12:20.522480 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:12:20.522480 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 12:12:20.522480 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 12:12:20.522480 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 12:12:20.522480 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 12:12:20.523442 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:12:20.523442 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:12:20.523442 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:12:20.523442 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:12:20.523442 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:12:20.523442 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:12:20.523442 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:12:20.523442 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application with Lock( Sep 15 12:12:20.523442 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:12:20.523442 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:12:20.523442 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:12:20.523442 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:12:20.523442 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:12:20.523442 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:12:20.523442 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:12:20.523442 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:12:20.523442 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 12:12:20.529200 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 12:12:20.529200 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 12:12:20.529200 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 15 12:12:20.529200 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 515/1038] 10.222.0.15 () {56 vars in 1108 bytes} [Thu Sep 15 12:12:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 15 12:12:20.529200 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a285b92-f8fa-4aa5-94b2-783d6c5a5988 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:12:20.554615 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af511f8-6836-450f-8122-57da42cf195f None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:12:20.591047 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8cce0f-1bd6-4a03-9d4b-fb170bdb691a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:12:20.591656 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8cce0f-1bd6-4a03-9d4b-fb170bdb691a None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:12:20.591943 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8cce0f-1bd6-4a03-9d4b-fb170bdb691a None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:12:20.592217 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8cce0f-1bd6-4a03-9d4b-fb170bdb691a None admin] PATH_INFO: `/v3/services` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:12:20.592874 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8cce0f-1bd6-4a03-9d4b-fb170bdb691a None admin] RBAC: Adding query filter params (type=image) {{(pid=81449) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:12:20.593175 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8cce0f-1bd6-4a03-9d4b-fb170bdb691a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:12:20.594532 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8cce0f-1bd6-4a03-9d4b-fb170bdb691a None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:12:20.599368 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 524/1039] 10.222.0.15 () {56 vars in 1108 bytes} [Thu Sep 15 12:12:20 2022] GET /identity/v3/services?type=image => generated 350 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 12:12:20.605996 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8cce0f-1bd6-4a03-9d4b-fb170bdb691a None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:12:20.641794 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a285b92-f8fa-4aa5-94b2-783d6c5a5988 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:12:20.667080 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:12:20.677800 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3fb4eaf-a545-4578-9c43-12c2790171a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:12:20.678347 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fb4eaf-a545-4578-9c43-12c2790171a1 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:12:20.678622 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fb4eaf-a545-4578-9c43-12c2790171a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:12:20.678899 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fb4eaf-a545-4578-9c43-12c2790171a1 None admin] PATH_INFO: `/v3/domains` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:12:20.679681 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3fb4eaf-a545-4578-9c43-12c2790171a1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:12:20.679980 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3fb4eaf-a545-4578-9c43-12c2790171a1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:12:20.682143 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3fb4eaf-a545-4578-9c43-12c2790171a1 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:12:20.695996 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 516/1040] 10.222.0.15 () {56 vars in 1109 bytes} [Thu Sep 15 12:12:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 12:12:20.701575 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3fb4eaf-a545-4578-9c43-12c2790171a1 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:12:20.709001 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:12:20.713876 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8cce0f-1bd6-4a03-9d4b-fb170bdb691a None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:12:20.749771 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5007a88-2504-4948-8e8a-114991e08ea1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:12:20.750368 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5007a88-2504-4948-8e8a-114991e08ea1 None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:12:20.750649 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5007a88-2504-4948-8e8a-114991e08ea1 None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:12:20.750917 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5007a88-2504-4948-8e8a-114991e08ea1 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:12:20.751578 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5007a88-2504-4948-8e8a-114991e08ea1 None admin] RBAC: Adding query filter params (interface=public, service_id=bbb5bc3e150a4f4192ca1e060b123fad, region_id=RegionOne) {{(pid=81449) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:12:20.752064 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5007a88-2504-4948-8e8a-114991e08ea1 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:12:20.753872 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5007a88-2504-4948-8e8a-114991e08ea1 None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:12:20.763382 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 525/1041] 10.222.0.15 () {56 vars in 1251 bytes} [Thu Sep 15 12:12:20 2022] GET /identity/v3/endpoints?service_id=bbb5bc3e150a4f4192ca1e060b123fad&interface=public®ion_id=RegionOne => generated 194 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 12:12:20.764353 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5007a88-2504-4948-8e8a-114991e08ea1 None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:12:20.813457 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3fb4eaf-a545-4578-9c43-12c2790171a1 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:12:20.850597 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ec5b40-ffc2-46d0-9f76-203197553767 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:12:20.851246 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ec5b40-ffc2-46d0-9f76-203197553767 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:12:20.851530 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ec5b40-ffc2-46d0-9f76-203197553767 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:12:20.851835 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ec5b40-ffc2-46d0-9f76-203197553767 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:12:20.859589 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ec5b40-ffc2-46d0-9f76-203197553767 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:12:20.861040 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ec5b40-ffc2-46d0-9f76-203197553767 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:12:20.867795 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5007a88-2504-4948-8e8a-114991e08ea1 None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:12:20.872258 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application [None req-66ec5b40-ffc2-46d0-9f76-203197553767 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 12:12:20.872258 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:12:20.872258 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:12:20.872258 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:12:20.872258 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:12:20.872258 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:12:20.872258 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:12:20.872258 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:12:20.872258 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:12:20.872258 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:12:20.872258 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:12:20.872258 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:12:20.872258 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 15 12:12:20.872258 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 15 12:12:20.872258 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 15 12:12:20.873387 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 15 12:12:20.873387 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:12:20.873387 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:12:20.873387 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:12:20.873387 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:12:20.873387 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:12:20.873387 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:12:20.873387 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:12:20.873387 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application with Lock( Sep 15 12:12:20.873387 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:12:20.873387 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:12:20.873387 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:12:20.873387 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:12:20.873387 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:12:20.873387 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:12:20.873387 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:12:20.873387 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:12:20.874321 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 15 12:12:20.874321 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 15 12:12:20.874321 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 15 12:12:20.874321 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 15 12:12:20.874321 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 15 12:12:20.874321 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 15 12:12:20.874321 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 12:12:20.874321 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 15 12:12:20.874321 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 517/1042] 10.222.0.15 () {56 vars in 1111 bytes} [Thu Sep 15 12:12:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 15 12:12:20.879621 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ec5b40-ffc2-46d0-9f76-203197553767 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:12:20.907664 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a680879-7c2e-4665-82db-d60d3fe5cf90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:12:20.908283 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a680879-7c2e-4665-82db-d60d3fe5cf90 None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:12:20.908590 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a680879-7c2e-4665-82db-d60d3fe5cf90 None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:12:20.908941 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a680879-7c2e-4665-82db-d60d3fe5cf90 None admin] PATH_INFO: `/v3/services` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:12:20.909550 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a680879-7c2e-4665-82db-d60d3fe5cf90 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:12:20.910916 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a680879-7c2e-4665-82db-d60d3fe5cf90 None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:12:20.924789 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 526/1043] 10.222.0.15 () {56 vars in 1087 bytes} [Thu Sep 15 12:12:20 2022] GET /identity/v3/services => generated 1957 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 15 12:12:20.939356 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:12:21.004849 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ec5b40-ffc2-46d0-9f76-203197553767 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:12:21.041007 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a24892-6267-4e9e-bdc6-8b7a60362601 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:12:21.041638 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a24892-6267-4e9e-bdc6-8b7a60362601 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:12:21.041925 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a24892-6267-4e9e-bdc6-8b7a60362601 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:12:21.042181 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a24892-6267-4e9e-bdc6-8b7a60362601 None admin] PATH_INFO: `/v3/projects` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:12:21.042875 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a24892-6267-4e9e-bdc6-8b7a60362601 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:12:21.043178 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a24892-6267-4e9e-bdc6-8b7a60362601 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:12:21.044606 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a24892-6267-4e9e-bdc6-8b7a60362601 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:12:21.063340 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 518/1044] 10.222.0.15 () {56 vars in 1148 bytes} [Thu Sep 15 12:12:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 12:12:21.065100 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a680879-7c2e-4665-82db-d60d3fe5cf90 None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:12:21.095197 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:12:21.174623 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a680879-7c2e-4665-82db-d60d3fe5cf90 None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:12:21.210547 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03a9774-72c6-4e7f-aa7f-8ad429c7d0d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:12:21.211495 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03a9774-72c6-4e7f-aa7f-8ad429c7d0d8 None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:12:21.212000 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03a9774-72c6-4e7f-aa7f-8ad429c7d0d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:12:21.212486 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03a9774-72c6-4e7f-aa7f-8ad429c7d0d8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:12:21.213442 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03a9774-72c6-4e7f-aa7f-8ad429c7d0d8 None admin] RBAC: Adding query filter params (role.id=c0320da65e0b4cdd8924c20a99b978af, scope.project.id=540a32ef88764e8c8d9201ae22f6ddaf, user.id=eface89e261840a2abb4206be26455c7) {{(pid=81449) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:12:21.213959 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03a9774-72c6-4e7f-aa7f-8ad429c7d0d8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:12:21.215651 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03a9774-72c6-4e7f-aa7f-8ad429c7d0d8 None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:12:21.222186 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 527/1045] 10.222.0.15 () {56 vars in 1374 bytes} [Thu Sep 15 12:12:21 2022] GET /identity/v3/role_assignments?user.id=eface89e261840a2abb4206be26455c7&scope.project.id=540a32ef88764e8c8d9201ae22f6ddaf&role.id=c0320da65e0b4cdd8924c20a99b978af => generated 259 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:12:21.763365 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd3f7fc-517e-4090-9e4a-b0d5c10e0808 None None] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:12:21.763883 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd3f7fc-517e-4090-9e4a-b0d5c10e0808 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:12:21.763962 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd3f7fc-517e-4090-9e4a-b0d5c10e0808 None None] PATH_INFO: `/` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:12:21.765296 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 519/1046] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 12:12:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:12:21.770529 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbdb222-b397-49f0-9fad-17b0a54e116c None None] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:12:21.770918 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbdb222-b397-49f0-9fad-17b0a54e116c None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:12:21.771113 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fbdb222-b397-49f0-9fad-17b0a54e116c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:12:21.789475 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.auth.core [None req-9fbdb222-b397-49f0-9fad-17b0a54e116c None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81449) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:12:21.790657 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-9fbdb222-b397-49f0-9fad-17b0a54e116c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:12:21.860817 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 528/1047] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 12:12:21 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 15 12:12:22.129662 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd3f7fc-517e-4090-9e4a-b0d5c10e0808 None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:12:22.231804 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd3f7fc-517e-4090-9e4a-b0d5c10e0808 None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:12:22.240121 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:12:22.273863 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df50ea5-7982-46e9-a972-b285ffa7eefb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:12:22.274490 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df50ea5-7982-46e9-a972-b285ffa7eefb None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:12:22.274762 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df50ea5-7982-46e9-a972-b285ffa7eefb None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:12:22.275025 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df50ea5-7982-46e9-a972-b285ffa7eefb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:12:22.284875 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6df50ea5-7982-46e9-a972-b285ffa7eefb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 12:12:22.289571 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application [None req-6df50ea5-7982-46e9-a972-b285ffa7eefb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 15 12:12:22.289571 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:12:22.289571 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:12:22.289571 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:12:22.289571 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:12:22.289571 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:12:22.289571 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:12:22.289571 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:12:22.289571 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:12:22.289571 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:12:22.289571 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:12:22.289571 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:12:22.289571 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 12:12:22.289571 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 12:12:22.289571 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 12:12:22.289571 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 12:12:22.290510 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 12:12:22.290510 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 12:12:22.290510 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 12:12:22.290510 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 12:12:22.290510 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:12:22.290510 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:12:22.290510 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:12:22.290510 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:12:22.290510 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:12:22.290510 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:12:22.290510 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:12:22.290510 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application with Lock( Sep 15 12:12:22.290510 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:12:22.290510 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:12:22.290510 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:12:22.290510 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:12:22.291413 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:12:22.291413 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:12:22.291413 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:12:22.291413 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:12:22.291413 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 12:12:22.291413 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 12:12:22.291413 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 12:12:22.291413 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 12:12:22.291413 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 12:12:22.291413 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 12:12:22.291413 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 15 12:12:22.291413 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 15 12:12:22.291413 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 520/1048] 10.222.0.15 () {56 vars in 1096 bytes} [Thu Sep 15 12:12:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:12:22.298070 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbdb222-b397-49f0-9fad-17b0a54e116c None None] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:12:22.319687 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:12:22.400182 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fbdb222-b397-49f0-9fad-17b0a54e116c None None] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:12:22.433742 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f4a052-e7ad-41f3-8c96-a2726f12d7db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:12:22.434355 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f4a052-e7ad-41f3-8c96-a2726f12d7db None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:12:22.434727 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f4a052-e7ad-41f3-8c96-a2726f12d7db None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:12:22.435006 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f4a052-e7ad-41f3-8c96-a2726f12d7db None admin] PATH_INFO: `/v3/roles` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:12:22.435661 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f4a052-e7ad-41f3-8c96-a2726f12d7db None admin] RBAC: Adding query filter params (name=admin) {{(pid=81449) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:12:22.435968 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f4a052-e7ad-41f3-8c96-a2726f12d7db None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:12:22.437356 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f4a052-e7ad-41f3-8c96-a2726f12d7db None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:12:22.449091 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 529/1049] 10.222.0.15 () {56 vars in 1099 bytes} [Thu Sep 15 12:12:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:12:22.451752 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df50ea5-7982-46e9-a972-b285ffa7eefb None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:12:22.550326 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6df50ea5-7982-46e9-a972-b285ffa7eefb None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:12:22.589510 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d933bca5-9c3a-409e-8e60-4b422ddbdadb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:12:22.590153 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d933bca5-9c3a-409e-8e60-4b422ddbdadb None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:12:22.590430 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d933bca5-9c3a-409e-8e60-4b422ddbdadb None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:12:22.590700 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d933bca5-9c3a-409e-8e60-4b422ddbdadb None admin] PATH_INFO: `/v3/users/2dbccf3842964c0ea8543e21f82cc76d` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:12:22.604506 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d933bca5-9c3a-409e-8e60-4b422ddbdadb None admin] RBAC: Authorizing `identity:get_user(user_id=2dbccf3842964c0ea8543e21f82cc76d)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:12:22.606007 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d933bca5-9c3a-409e-8e60-4b422ddbdadb None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:12:22.607790 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 521/1050] 10.222.0.15 () {56 vars in 1177 bytes} [Thu Sep 15 12:12:22 2022] GET /identity/v3/users/2dbccf3842964c0ea8543e21f82cc76d => generated 285 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:12:22.613403 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f4a052-e7ad-41f3-8c96-a2726f12d7db None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:12:22.697012 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:12:22.716180 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f4a052-e7ad-41f3-8c96-a2726f12d7db None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:12:22.749528 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b33230-8940-4084-9e1c-7da96d0ceda1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:12:22.750136 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b33230-8940-4084-9e1c-7da96d0ceda1 None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:12:22.750414 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b33230-8940-4084-9e1c-7da96d0ceda1 None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:12:22.750685 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b33230-8940-4084-9e1c-7da96d0ceda1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:12:22.818358 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b33230-8940-4084-9e1c-7da96d0ceda1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:12:22.819804 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b33230-8940-4084-9e1c-7da96d0ceda1 None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:12:22.902781 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:12:22.923084 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 530/1051] 10.222.0.15 () {58 vars in 1282 bytes} [Thu Sep 15 12:12:22 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 15 12:12:22.928833 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d933bca5-9c3a-409e-8e60-4b422ddbdadb None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:12:22.957226 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:12:23.041187 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d933bca5-9c3a-409e-8e60-4b422ddbdadb None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:12:23.075305 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcea80ef-282d-4ed8-aa27-4a0eb0a07bc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:12:23.076217 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcea80ef-282d-4ed8-aa27-4a0eb0a07bc3 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:12:23.076677 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcea80ef-282d-4ed8-aa27-4a0eb0a07bc3 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:12:23.077154 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcea80ef-282d-4ed8-aa27-4a0eb0a07bc3 None admin] PATH_INFO: `/v3/projects/b7fb192a30504033bcad23ae818e1e8c` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:12:23.078455 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcea80ef-282d-4ed8-aa27-4a0eb0a07bc3 None admin] RBAC: Authorizing `identity:get_project(project_id=b7fb192a30504033bcad23ae818e1e8c)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:12:23.080167 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcea80ef-282d-4ed8-aa27-4a0eb0a07bc3 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:12:23.082045 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 522/1052] 10.222.0.15 () {56 vars in 1186 bytes} [Thu Sep 15 12:12:22 2022] GET /identity/v3/projects/b7fb192a30504033bcad23ae818e1e8c => generated 311 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:12:23.088210 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b33230-8940-4084-9e1c-7da96d0ceda1 None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:12:23.120637 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a23d44-ba43-4b8d-a544-a31c3028d433 None None] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:12:23.121254 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a23d44-ba43-4b8d-a544-a31c3028d433 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:12:23.121707 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a23d44-ba43-4b8d-a544-a31c3028d433 None None] PATH_INFO: `/` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:12:23.123452 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 523/1053] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 12:12:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:12:23.128098 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e514e2-6fe3-4796-b789-82dddbeb5151 None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:12:23.128577 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e514e2-6fe3-4796-b789-82dddbeb5151 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:12:23.129048 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e514e2-6fe3-4796-b789-82dddbeb5151 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:12:23.149567 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-a2e514e2-6fe3-4796-b789-82dddbeb5151 None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:12:23.150955 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-a2e514e2-6fe3-4796-b789-82dddbeb5151 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:12:23.193300 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b33230-8940-4084-9e1c-7da96d0ceda1 None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:12:23.226645 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 524/1054] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 12:12:23 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 15 12:12:23.231510 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4687b70b-a1ca-4d05-ad57-4d80b792ca04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:12:23.232120 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4687b70b-a1ca-4d05-ad57-4d80b792ca04 None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:12:23.232399 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4687b70b-a1ca-4d05-ad57-4d80b792ca04 None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:12:23.232667 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4687b70b-a1ca-4d05-ad57-4d80b792ca04 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:12:23.233422 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4687b70b-a1ca-4d05-ad57-4d80b792ca04 None admin] RBAC: Adding query filter params (role.id=9c769b5c82884766b2c510b1f0f1364c, scope.project.id=b7fb192a30504033bcad23ae818e1e8c, user.id=2dbccf3842964c0ea8543e21f82cc76d) {{(pid=81449) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:12:23.233730 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4687b70b-a1ca-4d05-ad57-4d80b792ca04 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:12:23.235212 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4687b70b-a1ca-4d05-ad57-4d80b792ca04 None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:12:23.241190 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 531/1055] 10.222.0.15 () {56 vars in 1374 bytes} [Thu Sep 15 12:12:23 2022] GET /identity/v3/role_assignments?user.id=2dbccf3842964c0ea8543e21f82cc76d&scope.project.id=b7fb192a30504033bcad23ae818e1e8c&role.id=9c769b5c82884766b2c510b1f0f1364c => generated 259 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:12:23.401517 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f9d661-5942-4b64-ad21-9ef404d21766 None None] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:12:23.401854 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f9d661-5942-4b64-ad21-9ef404d21766 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:12:23.402083 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f9d661-5942-4b64-ad21-9ef404d21766 None None] PATH_INFO: `/` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:12:23.403432 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 525/1056] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 12:12:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:12:23.409647 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb26777-b105-437f-96b5-474329acb868 None None] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:12:23.409936 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb26777-b105-437f-96b5-474329acb868 None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:12:23.410208 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb26777-b105-437f-96b5-474329acb868 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:12:23.428340 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.auth.core [None req-3cb26777-b105-437f-96b5-474329acb868 None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81449) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:12:23.429555 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-3cb26777-b105-437f-96b5-474329acb868 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:12:23.473575 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:12:23.499598 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 532/1057] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 12:12:23 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 15 12:12:23.500408 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f9d661-5942-4b64-ad21-9ef404d21766 None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:12:23.524778 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:12:23.602311 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f9d661-5942-4b64-ad21-9ef404d21766 None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:12:23.635018 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1716c8be-e249-4e28-b53b-83fd66244c51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:12:23.635643 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1716c8be-e249-4e28-b53b-83fd66244c51 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:12:23.635922 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1716c8be-e249-4e28-b53b-83fd66244c51 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:12:23.636187 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1716c8be-e249-4e28-b53b-83fd66244c51 None admin] PATH_INFO: `/v3/services/image` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:12:23.636766 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1716c8be-e249-4e28-b53b-83fd66244c51 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:12:23.638117 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1716c8be-e249-4e28-b53b-83fd66244c51 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:12:23.647190 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application [None req-1716c8be-e249-4e28-b53b-83fd66244c51 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Sep 15 12:12:23.647190 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:12:23.647190 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 15 12:12:23.647190 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 15 12:12:23.647190 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 15 12:12:23.647190 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 15 12:12:23.647190 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 15 12:12:23.647190 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 15 12:12:23.647190 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 15 12:12:23.647190 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 15 12:12:23.647190 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 15 12:12:23.647190 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 15 12:12:23.647190 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:12:23.647190 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:12:23.647190 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:12:23.647190 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:12:23.647190 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:12:23.648182 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:12:23.648182 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:12:23.648182 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:12:23.648182 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:12:23.648182 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:12:23.648182 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:12:23.648182 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 15 12:12:23.648182 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 15 12:12:23.648182 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 15 12:12:23.648182 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 15 12:12:23.648182 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:12:23.648182 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:12:23.648182 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:12:23.648182 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:12:23.648182 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:12:23.648182 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:12:23.649047 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:12:23.649047 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application with Lock( Sep 15 12:12:23.649047 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:12:23.649047 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:12:23.649047 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:12:23.649047 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:12:23.649047 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:12:23.649047 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:12:23.649047 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:12:23.649047 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:12:23.649047 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 15 12:12:23.649047 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 15 12:12:23.649047 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Sep 15 12:12:23.649047 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 15 12:12:23.649047 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 526/1058] 10.222.0.15 () {56 vars in 1105 bytes} [Thu Sep 15 12:12:23 2022] GET /identity/v3/services/image => generated 86 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 15 12:12:23.654313 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb26777-b105-437f-96b5-474329acb868 None None] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:12:23.757136 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb26777-b105-437f-96b5-474329acb868 None None] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:12:23.766411 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1716c8be-e249-4e28-b53b-83fd66244c51 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:12:23.791682 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e6369f-d742-4048-b62a-919044c7f1f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:12:23.792290 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e6369f-d742-4048-b62a-919044c7f1f4 None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:12:23.792567 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e6369f-d742-4048-b62a-919044c7f1f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:12:23.792862 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e6369f-d742-4048-b62a-919044c7f1f4 None admin] PATH_INFO: `/v3/services` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:12:23.793485 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e6369f-d742-4048-b62a-919044c7f1f4 None admin] RBAC: Adding query filter params (name=image) {{(pid=81449) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:12:23.793791 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e6369f-d742-4048-b62a-919044c7f1f4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:12:23.795139 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e6369f-d742-4048-b62a-919044c7f1f4 None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:12:23.800427 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 533/1059] 10.222.0.15 () {56 vars in 1108 bytes} [Thu Sep 15 12:12:23 2022] GET /identity/v3/services?name=image => generated 122 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:12:23.806413 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e6369f-d742-4048-b62a-919044c7f1f4 None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:12:23.850650 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:12:23.860636 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:12:23.878133 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1716c8be-e249-4e28-b53b-83fd66244c51 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:12:23.912409 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f29c88a-6169-445a-874d-a77838bfaf13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:12:23.913071 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f29c88a-6169-445a-874d-a77838bfaf13 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:12:23.913350 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f29c88a-6169-445a-874d-a77838bfaf13 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:12:23.913611 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f29c88a-6169-445a-874d-a77838bfaf13 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:12:23.921863 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e6369f-d742-4048-b62a-919044c7f1f4 None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:12:23.926132 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f29c88a-6169-445a-874d-a77838bfaf13 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 12:12:23.930921 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application [None req-8f29c88a-6169-445a-874d-a77838bfaf13 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 15 12:12:23.930921 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:12:23.930921 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:12:23.930921 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:12:23.930921 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:12:23.930921 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:12:23.930921 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:12:23.930921 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:12:23.930921 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:12:23.930921 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:12:23.930921 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:12:23.930921 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:12:23.930921 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 12:12:23.930921 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 12:12:23.930921 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 12:12:23.930921 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 12:12:23.933292 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 12:12:23.933292 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 12:12:23.933292 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 12:12:23.933292 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 12:12:23.933292 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:12:23.933292 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:12:23.933292 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:12:23.933292 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:12:23.933292 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:12:23.933292 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:12:23.933292 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:12:23.933292 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application with Lock( Sep 15 12:12:23.933292 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:12:23.933292 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:12:23.933292 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:12:23.933292 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:12:23.934361 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:12:23.934361 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:12:23.934361 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:12:23.934361 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:12:23.934361 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 12:12:23.934361 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 12:12:23.934361 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 12:12:23.934361 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 12:12:23.934361 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 12:12:23.934361 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 12:12:23.934361 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 15 12:12:23.934361 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 15 12:12:23.934361 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 527/1060] 10.222.0.15 () {56 vars in 1120 bytes} [Thu Sep 15 12:12:23 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 15 12:12:23.937866 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f29c88a-6169-445a-874d-a77838bfaf13 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:12:23.965127 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5d9544-a90b-42ec-90d4-7fdd6a558e60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:12:23.965743 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5d9544-a90b-42ec-90d4-7fdd6a558e60 None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:12:23.966065 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5d9544-a90b-42ec-90d4-7fdd6a558e60 None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:12:23.966344 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5d9544-a90b-42ec-90d4-7fdd6a558e60 None admin] PATH_INFO: `/v3/services` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:12:23.967081 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5d9544-a90b-42ec-90d4-7fdd6a558e60 None admin] RBAC: Adding query filter params (type=image) {{(pid=81449) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:12:23.967567 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5d9544-a90b-42ec-90d4-7fdd6a558e60 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:12:23.968961 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5d9544-a90b-42ec-90d4-7fdd6a558e60 None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:12:23.979320 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 534/1061] 10.222.0.15 () {56 vars in 1108 bytes} [Thu Sep 15 12:12:23 2022] GET /identity/v3/services?type=image => generated 350 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 12:12:23.980420 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5d9544-a90b-42ec-90d4-7fdd6a558e60 None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:12:24.050403 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f29c88a-6169-445a-874d-a77838bfaf13 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:12:24.083527 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966ffa99-3b04-4745-b290-42dfbdde0f09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:12:24.084137 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966ffa99-3b04-4745-b290-42dfbdde0f09 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:12:24.084427 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966ffa99-3b04-4745-b290-42dfbdde0f09 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:12:24.084697 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-966ffa99-3b04-4745-b290-42dfbdde0f09 None admin] PATH_INFO: `/v3/roles` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:12:24.085386 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-966ffa99-3b04-4745-b290-42dfbdde0f09 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:12:24.085679 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-966ffa99-3b04-4745-b290-42dfbdde0f09 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:12:24.087089 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-966ffa99-3b04-4745-b290-42dfbdde0f09 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:12:24.088513 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5d9544-a90b-42ec-90d4-7fdd6a558e60 None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:12:24.091110 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:12:24.100323 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 528/1062] 10.222.0.15 () {56 vars in 1115 bytes} [Thu Sep 15 12:12:23 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:12:24.103431 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966ffa99-3b04-4745-b290-42dfbdde0f09 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:12:24.115937 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:12:24.133667 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a8dc7a-f807-4579-adb8-6cd76e23c7bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:12:24.134265 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a8dc7a-f807-4579-adb8-6cd76e23c7bf None admin] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:12:24.134541 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a8dc7a-f807-4579-adb8-6cd76e23c7bf None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:12:24.134811 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a8dc7a-f807-4579-adb8-6cd76e23c7bf None admin] PATH_INFO: `/v3/endpoints` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:12:24.135455 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a8dc7a-f807-4579-adb8-6cd76e23c7bf None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:12:24.136831 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a8dc7a-f807-4579-adb8-6cd76e23c7bf None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:12:24.168785 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 535/1063] 10.222.0.15 () {60 vars in 1144 bytes} [Thu Sep 15 12:12:23 2022] POST /identity/v3/endpoints => generated 327 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 12:12:24.212216 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-966ffa99-3b04-4745-b290-42dfbdde0f09 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:12:24.256308 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e5be19-14b3-45d3-ab36-e160e1b8ec5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:12:24.256985 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e5be19-14b3-45d3-ab36-e160e1b8ec5e None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:12:24.257262 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e5be19-14b3-45d3-ab36-e160e1b8ec5e None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:12:24.257528 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e5be19-14b3-45d3-ab36-e160e1b8ec5e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:12:24.268972 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e5be19-14b3-45d3-ab36-e160e1b8ec5e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:12:24.270435 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e5be19-14b3-45d3-ab36-e160e1b8ec5e None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:12:24.285609 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application [None req-41e5be19-14b3-45d3-ab36-e160e1b8ec5e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 12:12:24.285609 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:12:24.285609 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:12:24.285609 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:12:24.285609 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:12:24.285609 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:12:24.285609 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:12:24.285609 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:12:24.285609 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:12:24.285609 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:12:24.285609 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:12:24.285609 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:12:24.285609 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 12:12:24.285609 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 12:12:24.285609 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 12:12:24.285609 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 12:12:24.286634 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:12:24.286634 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:12:24.286634 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:12:24.286634 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:12:24.286634 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:12:24.286634 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:12:24.286634 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:12:24.286634 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application with Lock( Sep 15 12:12:24.286634 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:12:24.286634 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:12:24.286634 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:12:24.286634 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:12:24.286634 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:12:24.286634 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:12:24.286634 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:12:24.286634 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:12:24.286634 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 12:12:24.287578 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 12:12:24.287578 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 12:12:24.287578 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 15 12:12:24.287578 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 529/1064] 10.222.0.15 () {56 vars in 1108 bytes} [Thu Sep 15 12:12:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 15 12:12:24.294072 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a8dc7a-f807-4579-adb8-6cd76e23c7bf None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:12:24.401656 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a8dc7a-f807-4579-adb8-6cd76e23c7bf None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:12:24.439989 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d42962-df32-426f-90e7-7b29339b7ec7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:12:24.440597 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d42962-df32-426f-90e7-7b29339b7ec7 None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:12:24.440907 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d42962-df32-426f-90e7-7b29339b7ec7 None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:12:24.441182 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d42962-df32-426f-90e7-7b29339b7ec7 None admin] PATH_INFO: `/v3/domains` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:12:24.441816 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d42962-df32-426f-90e7-7b29339b7ec7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81449) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:12:24.442119 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d42962-df32-426f-90e7-7b29339b7ec7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:12:24.443478 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d42962-df32-426f-90e7-7b29339b7ec7 None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:12:24.455199 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 536/1065] 10.222.0.15 () {56 vars in 1109 bytes} [Thu Sep 15 12:12:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:12:24.461562 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e5be19-14b3-45d3-ab36-e160e1b8ec5e None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:12:24.521471 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:12:24.567237 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e5be19-14b3-45d3-ab36-e160e1b8ec5e None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:12:24.602439 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-defdde47-fb14-4a5e-b9c5-29bb3cc41b17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:12:24.603075 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defdde47-fb14-4a5e-b9c5-29bb3cc41b17 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:12:24.603349 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defdde47-fb14-4a5e-b9c5-29bb3cc41b17 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:12:24.603868 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defdde47-fb14-4a5e-b9c5-29bb3cc41b17 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:12:24.611860 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-defdde47-fb14-4a5e-b9c5-29bb3cc41b17 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:12:24.613304 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-defdde47-fb14-4a5e-b9c5-29bb3cc41b17 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:12:24.624154 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application [None req-defdde47-fb14-4a5e-b9c5-29bb3cc41b17 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 15 12:12:24.624154 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:12:24.624154 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:12:24.624154 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:12:24.624154 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:12:24.624154 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:12:24.624154 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:12:24.624154 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:12:24.624154 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:12:24.624154 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:12:24.624154 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:12:24.624154 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:12:24.624154 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 15 12:12:24.624154 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 15 12:12:24.624154 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 15 12:12:24.624154 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 15 12:12:24.625135 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:12:24.625135 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:12:24.625135 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 15 12:12:24.625135 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 12:12:24.625135 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 15 12:12:24.625135 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 12:12:24.625135 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:12:24.625135 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:12:24.625135 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:12:24.625135 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:12:24.625135 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:12:24.625135 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application with Lock( Sep 15 12:12:24.625135 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:12:24.625135 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:12:24.625135 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:12:24.625135 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:12:24.625135 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:12:24.625135 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:12:24.626094 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:12:24.626094 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:12:24.626094 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 15 12:12:24.626094 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 15 12:12:24.626094 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 15 12:12:24.626094 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 15 12:12:24.626094 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 15 12:12:24.626094 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 15 12:12:24.626094 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 15 12:12:24.626094 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 15 12:12:24.628823 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 530/1066] 10.222.0.15 () {56 vars in 1093 bytes} [Thu Sep 15 12:12:24 2022] GET /identity/v3/users/nova => generated 82 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:12:24.631476 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d42962-df32-426f-90e7-7b29339b7ec7 None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:12:24.654544 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:12:24.735625 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d42962-df32-426f-90e7-7b29339b7ec7 None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:12:24.769917 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249f3495-ab1f-4df8-b64c-206c0c1cd05a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:12:24.770530 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249f3495-ab1f-4df8-b64c-206c0c1cd05a None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:12:24.770810 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249f3495-ab1f-4df8-b64c-206c0c1cd05a None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:12:24.771080 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249f3495-ab1f-4df8-b64c-206c0c1cd05a None admin] PATH_INFO: `/v3/users` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:12:24.771799 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249f3495-ab1f-4df8-b64c-206c0c1cd05a None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81449) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:12:24.772097 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249f3495-ab1f-4df8-b64c-206c0c1cd05a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:12:24.773542 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249f3495-ab1f-4df8-b64c-206c0c1cd05a None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:12:24.790568 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 537/1067] 10.222.0.15 () {56 vars in 1133 bytes} [Thu Sep 15 12:12:24 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:12:24.796430 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-defdde47-fb14-4a5e-b9c5-29bb3cc41b17 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:12:24.902020 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-defdde47-fb14-4a5e-b9c5-29bb3cc41b17 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:12:24.918191 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:12:24.937245 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c65589-c8b7-4717-bd67-a8960ac524a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:12:24.937965 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c65589-c8b7-4717-bd67-a8960ac524a3 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:12:24.938162 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c65589-c8b7-4717-bd67-a8960ac524a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:12:24.938429 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c65589-c8b7-4717-bd67-a8960ac524a3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:12:24.949740 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c65589-c8b7-4717-bd67-a8960ac524a3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:12:24.951168 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c65589-c8b7-4717-bd67-a8960ac524a3 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:12:24.968667 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application [None req-a3c65589-c8b7-4717-bd67-a8960ac524a3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 12:12:24.968667 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:12:24.968667 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:12:24.968667 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:12:24.968667 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:12:24.968667 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:12:24.968667 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:12:24.968667 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:12:24.968667 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:12:24.968667 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:12:24.968667 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:12:24.968667 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:12:24.968667 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 12:12:24.968667 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 12:12:24.968667 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 12:12:24.968667 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 12:12:24.969674 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:12:24.969674 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:12:24.969674 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:12:24.969674 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:12:24.969674 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:12:24.969674 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:12:24.969674 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:12:24.969674 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application with Lock( Sep 15 12:12:24.969674 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:12:24.969674 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:12:24.969674 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:12:24.969674 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:12:24.969674 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:12:24.969674 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:12:24.969674 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:12:24.969674 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:12:24.969674 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 12:12:24.970593 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 12:12:24.970593 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 12:12:24.970593 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 15 12:12:24.971434 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 531/1068] 10.222.0.15 () {56 vars in 1108 bytes} [Thu Sep 15 12:12:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:12:24.975578 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249f3495-ab1f-4df8-b64c-206c0c1cd05a None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:12:25.058292 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:12:25.083579 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249f3495-ab1f-4df8-b64c-206c0c1cd05a None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:12:25.118569 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3a146c-6cd8-4bb2-ad69-90384fd3ff9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:12:25.119179 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3a146c-6cd8-4bb2-ad69-90384fd3ff9e None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:12:25.119458 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3a146c-6cd8-4bb2-ad69-90384fd3ff9e None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:12:25.119724 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3a146c-6cd8-4bb2-ad69-90384fd3ff9e None admin] PATH_INFO: `/v3/domains` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:12:25.120356 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed3a146c-6cd8-4bb2-ad69-90384fd3ff9e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81449) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:12:25.120655 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed3a146c-6cd8-4bb2-ad69-90384fd3ff9e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:12:25.122030 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed3a146c-6cd8-4bb2-ad69-90384fd3ff9e None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:12:25.138151 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 538/1069] 10.222.0.15 () {56 vars in 1109 bytes} [Thu Sep 15 12:12:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 12:12:25.139297 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c65589-c8b7-4717-bd67-a8960ac524a3 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:12:25.238490 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c65589-c8b7-4717-bd67-a8960ac524a3 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:12:25.274684 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92fa8d39-6b57-4ee5-961f-177585bb6004 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:12:25.275345 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fa8d39-6b57-4ee5-961f-177585bb6004 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:12:25.275621 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fa8d39-6b57-4ee5-961f-177585bb6004 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:12:25.275892 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fa8d39-6b57-4ee5-961f-177585bb6004 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:12:25.283975 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92fa8d39-6b57-4ee5-961f-177585bb6004 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:12:25.285465 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92fa8d39-6b57-4ee5-961f-177585bb6004 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:12:25.296843 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application [None req-92fa8d39-6b57-4ee5-961f-177585bb6004 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 12:12:25.296843 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:12:25.296843 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:12:25.296843 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:12:25.296843 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:12:25.296843 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:12:25.296843 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:12:25.296843 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:12:25.296843 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:12:25.296843 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:12:25.296843 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:12:25.296843 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:12:25.296843 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 15 12:12:25.296843 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 15 12:12:25.296843 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 15 12:12:25.297732 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 15 12:12:25.297732 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:12:25.297732 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:12:25.297732 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:12:25.297732 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:12:25.297732 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:12:25.297732 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:12:25.297732 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:12:25.297732 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application with Lock( Sep 15 12:12:25.297732 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:12:25.297732 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:12:25.297732 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:12:25.297732 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:12:25.297732 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:12:25.297732 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:12:25.297732 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:12:25.297732 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:12:25.304067 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 15 12:12:25.304067 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 15 12:12:25.304067 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 15 12:12:25.304067 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 15 12:12:25.304067 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 15 12:12:25.304067 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 15 12:12:25.304067 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 12:12:25.304067 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 15 12:12:25.304067 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 532/1070] 10.222.0.15 () {56 vars in 1111 bytes} [Thu Sep 15 12:12:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 15 12:12:25.304578 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3a146c-6cd8-4bb2-ad69-90384fd3ff9e None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:12:25.370531 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ce02d7-bc5b-42b5-a586-0f945cb57411 None None] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:12:25.370907 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ce02d7-bc5b-42b5-a586-0f945cb57411 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:12:25.371262 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ce02d7-bc5b-42b5-a586-0f945cb57411 None None] PATH_INFO: `/` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:12:25.372655 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 533/1071] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 12:12:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 15 12:12:25.378066 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e15802-e2da-46fa-837b-f60bb1b71e4b None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:12:25.378341 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e15802-e2da-46fa-837b-f60bb1b71e4b None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:12:25.378609 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e15802-e2da-46fa-837b-f60bb1b71e4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:12:25.397301 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-c4e15802-e2da-46fa-837b-f60bb1b71e4b None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:12:25.398525 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-c4e15802-e2da-46fa-837b-f60bb1b71e4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:12:25.409337 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3a146c-6cd8-4bb2-ad69-90384fd3ff9e None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:12:25.422016 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:12:25.444108 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca30f1f9-f6a9-4767-8b7d-7dcd99f3b360 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:12:25.444726 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca30f1f9-f6a9-4767-8b7d-7dcd99f3b360 None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:12:25.445032 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca30f1f9-f6a9-4767-8b7d-7dcd99f3b360 None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:12:25.445304 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca30f1f9-f6a9-4767-8b7d-7dcd99f3b360 None admin] PATH_INFO: `/v3/projects` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:12:25.445991 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca30f1f9-f6a9-4767-8b7d-7dcd99f3b360 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81449) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:12:25.446292 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca30f1f9-f6a9-4767-8b7d-7dcd99f3b360 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:12:25.447710 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca30f1f9-f6a9-4767-8b7d-7dcd99f3b360 None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:12:25.456018 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:12:25.462525 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 539/1072] 10.222.0.15 () {56 vars in 1148 bytes} [Thu Sep 15 12:12:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:12:25.468327 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca30f1f9-f6a9-4767-8b7d-7dcd99f3b360 None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:12:25.474863 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 534/1073] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 12:12:25 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 15 12:12:25.565670 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca30f1f9-f6a9-4767-8b7d-7dcd99f3b360 None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:12:25.603261 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2c8bb1-d5e7-411a-a260-1ba6ff6f761d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:12:25.604175 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2c8bb1-d5e7-411a-a260-1ba6ff6f761d None admin] REQUEST_METHOD: `PUT` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:12:25.604454 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2c8bb1-d5e7-411a-a260-1ba6ff6f761d None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:12:25.604723 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2c8bb1-d5e7-411a-a260-1ba6ff6f761d None admin] PATH_INFO: `/v3/projects/540a32ef88764e8c8d9201ae22f6ddaf/users/eface89e261840a2abb4206be26455c7/roles/c0320da65e0b4cdd8924c20a99b978af` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:12:25.606926 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2c8bb1-d5e7-411a-a260-1ba6ff6f761d None admin] RBAC: Authorizing `identity:create_grant(project_id=540a32ef88764e8c8d9201ae22f6ddaf, user_id=eface89e261840a2abb4206be26455c7, role_id=c0320da65e0b4cdd8924c20a99b978af)` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:12:25.608371 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2c8bb1-d5e7-411a-a260-1ba6ff6f761d None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:12:25.621854 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 540/1074] 10.222.0.15 () {58 vars in 1439 bytes} [Thu Sep 15 12:12:25 2022] PUT /identity/v3/projects/540a32ef88764e8c8d9201ae22f6ddaf/users/eface89e261840a2abb4206be26455c7/roles/c0320da65e0b4cdd8924c20a99b978af => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 12:12:25.739693 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e15802-e2da-46fa-837b-f60bb1b71e4b None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:12:25.844536 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e15802-e2da-46fa-837b-f60bb1b71e4b None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:12:25.878795 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0015b4-300a-4f44-9bca-220d7aad705d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:12:25.879433 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0015b4-300a-4f44-9bca-220d7aad705d None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:12:25.879711 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0015b4-300a-4f44-9bca-220d7aad705d None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:12:25.879978 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0015b4-300a-4f44-9bca-220d7aad705d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:12:25.890052 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0015b4-300a-4f44-9bca-220d7aad705d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 12:12:25.895367 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application [None req-cd0015b4-300a-4f44-9bca-220d7aad705d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 15 12:12:25.895367 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:12:25.895367 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:12:25.895367 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:12:25.895367 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:12:25.895367 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:12:25.895367 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:12:25.895367 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:12:25.895367 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:12:25.895367 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:12:25.895367 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:12:25.895367 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:12:25.895367 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 12:12:25.895367 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 12:12:25.895367 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 12:12:25.895367 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 12:12:25.896331 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 12:12:25.896331 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 12:12:25.896331 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 12:12:25.896331 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 12:12:25.896331 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:12:25.896331 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:12:25.896331 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:12:25.896331 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:12:25.896331 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:12:25.896331 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:12:25.896331 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:12:25.896331 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application with Lock( Sep 15 12:12:25.896331 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:12:25.896331 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:12:25.896331 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:12:25.896331 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:12:25.897224 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:12:25.897224 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:12:25.897224 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:12:25.897224 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:12:25.897224 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 12:12:25.897224 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 12:12:25.897224 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 12:12:25.897224 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 12:12:25.897224 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 12:12:25.897224 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 12:12:25.897224 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 15 12:12:25.897224 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 15 12:12:25.897224 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 535/1075] 10.222.0.15 () {56 vars in 1096 bytes} [Thu Sep 15 12:12:25 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:12:25.903987 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2c8bb1-d5e7-411a-a260-1ba6ff6f761d None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:12:25.966988 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:12:26.016729 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2c8bb1-d5e7-411a-a260-1ba6ff6f761d None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:12:26.050335 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a857c1d8-ddfd-4a7b-9aca-d3ff468887d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:12:26.050941 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a857c1d8-ddfd-4a7b-9aca-d3ff468887d2 None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:12:26.051217 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a857c1d8-ddfd-4a7b-9aca-d3ff468887d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:12:26.051485 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a857c1d8-ddfd-4a7b-9aca-d3ff468887d2 None admin] PATH_INFO: `/v3/roles` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:12:26.052136 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a857c1d8-ddfd-4a7b-9aca-d3ff468887d2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81449) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:12:26.052431 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a857c1d8-ddfd-4a7b-9aca-d3ff468887d2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:12:26.053809 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a857c1d8-ddfd-4a7b-9aca-d3ff468887d2 None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:12:26.067094 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 541/1076] 10.222.0.15 () {56 vars in 1099 bytes} [Thu Sep 15 12:12:25 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:12:26.068248 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0015b4-300a-4f44-9bca-220d7aad705d None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:12:26.089628 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:12:26.172652 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0015b4-300a-4f44-9bca-220d7aad705d None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:12:26.207583 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8445b67-8eb0-41b8-9351-4a1bca38a48a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:12:26.208547 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8445b67-8eb0-41b8-9351-4a1bca38a48a None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:12:26.209006 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8445b67-8eb0-41b8-9351-4a1bca38a48a None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:12:26.209432 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8445b67-8eb0-41b8-9351-4a1bca38a48a None admin] PATH_INFO: `/v3/users/2dbccf3842964c0ea8543e21f82cc76d` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:12:26.210695 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8445b67-8eb0-41b8-9351-4a1bca38a48a None admin] RBAC: Authorizing `identity:get_user(user_id=2dbccf3842964c0ea8543e21f82cc76d)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:12:26.212283 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8445b67-8eb0-41b8-9351-4a1bca38a48a None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:12:26.214039 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 536/1077] 10.222.0.15 () {56 vars in 1177 bytes} [Thu Sep 15 12:12:26 2022] GET /identity/v3/users/2dbccf3842964c0ea8543e21f82cc76d => generated 285 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:12:26.219690 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a857c1d8-ddfd-4a7b-9aca-d3ff468887d2 None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:12:26.329330 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a857c1d8-ddfd-4a7b-9aca-d3ff468887d2 None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:12:26.350484 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:12:26.365607 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12de4649-e5a1-4cb8-9026-e6227ce245e8 None None] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:12:26.365897 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12de4649-e5a1-4cb8-9026-e6227ce245e8 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:12:26.366172 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12de4649-e5a1-4cb8-9026-e6227ce245e8 None None] PATH_INFO: `/` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:12:26.366674 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38fd8bd8-2699-422a-8772-81a209487f25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:12:26.367371 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fd8bd8-2699-422a-8772-81a209487f25 None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:12:26.367716 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 537/1078] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 12:12:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:12:26.367838 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fd8bd8-2699-422a-8772-81a209487f25 None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:12:26.368025 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fd8bd8-2699-422a-8772-81a209487f25 None admin] PATH_INFO: `/v3/projects/b7fb192a30504033bcad23ae818e1e8c` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:12:26.369238 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38fd8bd8-2699-422a-8772-81a209487f25 None admin] RBAC: Authorizing `identity:get_project(project_id=b7fb192a30504033bcad23ae818e1e8c)` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:12:26.370807 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38fd8bd8-2699-422a-8772-81a209487f25 None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:12:26.376784 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 542/1079] 10.222.0.15 () {56 vars in 1186 bytes} [Thu Sep 15 12:12:26 2022] GET /identity/v3/projects/b7fb192a30504033bcad23ae818e1e8c => generated 311 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 12:12:26.376940 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1493c3ab-4ce8-44f6-9ff2-ddf81d65bc30 None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:12:26.376940 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1493c3ab-4ce8-44f6-9ff2-ddf81d65bc30 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:12:26.376940 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1493c3ab-4ce8-44f6-9ff2-ddf81d65bc30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:12:26.379147 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38fd8bd8-2699-422a-8772-81a209487f25 None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:12:26.394026 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-1493c3ab-4ce8-44f6-9ff2-ddf81d65bc30 None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:12:26.395213 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-1493c3ab-4ce8-44f6-9ff2-ddf81d65bc30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:12:26.465607 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 538/1080] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 12:12:26 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 15 12:12:26.487896 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38fd8bd8-2699-422a-8772-81a209487f25 None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:12:26.525038 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93b3867-0dca-40dc-9bcb-807a5809549b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:12:26.525839 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93b3867-0dca-40dc-9bcb-807a5809549b None admin] REQUEST_METHOD: `PUT` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:12:26.526189 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93b3867-0dca-40dc-9bcb-807a5809549b None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:12:26.526529 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f93b3867-0dca-40dc-9bcb-807a5809549b None admin] PATH_INFO: `/v3/projects/b7fb192a30504033bcad23ae818e1e8c/users/2dbccf3842964c0ea8543e21f82cc76d/roles/9c769b5c82884766b2c510b1f0f1364c` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:12:26.528568 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f93b3867-0dca-40dc-9bcb-807a5809549b None admin] RBAC: Authorizing `identity:create_grant(project_id=b7fb192a30504033bcad23ae818e1e8c, user_id=2dbccf3842964c0ea8543e21f82cc76d, role_id=9c769b5c82884766b2c510b1f0f1364c)` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:12:26.530095 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f93b3867-0dca-40dc-9bcb-807a5809549b None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:12:26.552783 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 543/1081] 10.222.0.15 () {58 vars in 1439 bytes} [Thu Sep 15 12:12:26 2022] PUT /identity/v3/projects/b7fb192a30504033bcad23ae818e1e8c/users/2dbccf3842964c0ea8543e21f82cc76d/roles/9c769b5c82884766b2c510b1f0f1364c => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 12:12:26.743484 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1493c3ab-4ce8-44f6-9ff2-ddf81d65bc30 None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:12:26.761514 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:12:26.846624 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1493c3ab-4ce8-44f6-9ff2-ddf81d65bc30 None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:12:26.881034 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e39b25-1873-4d61-8ce8-62cab2071cb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:12:26.881650 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e39b25-1873-4d61-8ce8-62cab2071cb2 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:12:26.881926 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e39b25-1873-4d61-8ce8-62cab2071cb2 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:12:26.882196 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e39b25-1873-4d61-8ce8-62cab2071cb2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:12:26.949130 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e39b25-1873-4d61-8ce8-62cab2071cb2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:12:26.950564 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e39b25-1873-4d61-8ce8-62cab2071cb2 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:12:26.968695 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:12:27.055689 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 539/1082] 10.222.0.15 () {58 vars in 1282 bytes} [Thu Sep 15 12:12:26 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 15 12:12:27.061390 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93b3867-0dca-40dc-9bcb-807a5809549b None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:12:27.112703 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:12:27.166131 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f93b3867-0dca-40dc-9bcb-807a5809549b None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:12:27.200658 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4bf3bc-ff79-46f3-82fa-eb631a3c3f76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:12:27.201323 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4bf3bc-ff79-46f3-82fa-eb631a3c3f76 None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:12:27.201604 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4bf3bc-ff79-46f3-82fa-eb631a3c3f76 None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:12:27.201880 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4bf3bc-ff79-46f3-82fa-eb631a3c3f76 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:12:27.212562 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4bf3bc-ff79-46f3-82fa-eb631a3c3f76 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:12:27.214040 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4bf3bc-ff79-46f3-82fa-eb631a3c3f76 None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:12:27.228159 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application [None req-bb4bf3bc-ff79-46f3-82fa-eb631a3c3f76 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 12:12:27.228159 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:12:27.228159 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:12:27.228159 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:12:27.228159 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:12:27.228159 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:12:27.228159 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:12:27.228159 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:12:27.228159 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:12:27.228159 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:12:27.228159 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:12:27.228159 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:12:27.228159 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 12:12:27.228159 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 12:12:27.228159 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 12:12:27.228159 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 12:12:27.229437 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:12:27.229437 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:12:27.229437 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:12:27.229437 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:12:27.229437 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:12:27.229437 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:12:27.229437 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:12:27.229437 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application with Lock( Sep 15 12:12:27.229437 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:12:27.229437 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:12:27.229437 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:12:27.229437 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:12:27.229437 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:12:27.229437 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:12:27.229437 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:12:27.229437 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:12:27.229437 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 12:12:27.233928 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 12:12:27.233928 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 12:12:27.233928 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application Sep 15 12:12:27.233928 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 544/1083] 10.222.0.15 () {56 vars in 1108 bytes} [Thu Sep 15 12:12:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 15 12:12:27.237840 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e39b25-1873-4d61-8ce8-62cab2071cb2 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:12:27.338627 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e39b25-1873-4d61-8ce8-62cab2071cb2 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:12:27.355511 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:12:27.374769 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f980a2-a800-4db4-bec0-5b5be0d4ca97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:12:27.375382 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f980a2-a800-4db4-bec0-5b5be0d4ca97 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:12:27.375659 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f980a2-a800-4db4-bec0-5b5be0d4ca97 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:12:27.375927 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f980a2-a800-4db4-bec0-5b5be0d4ca97 None admin] PATH_INFO: `/v3/domains` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:12:27.376561 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f980a2-a800-4db4-bec0-5b5be0d4ca97 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:12:27.376890 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f980a2-a800-4db4-bec0-5b5be0d4ca97 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:12:27.378831 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f980a2-a800-4db4-bec0-5b5be0d4ca97 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:12:27.390248 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 540/1084] 10.222.0.15 () {56 vars in 1109 bytes} [Thu Sep 15 12:12:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:12:27.810951 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e532484-babe-40ea-b121-573e80ebbdaa None None] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:12:27.811244 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e532484-babe-40ea-b121-573e80ebbdaa None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:12:27.811517 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e532484-babe-40ea-b121-573e80ebbdaa None None] PATH_INFO: `/` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:12:27.812892 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 545/1085] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 12:12:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:12:27.818220 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e091b5aa-fac9-4eb9-86e7-ba8683d76e51 None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:12:27.818578 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e091b5aa-fac9-4eb9-86e7-ba8683d76e51 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:12:27.818917 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e091b5aa-fac9-4eb9-86e7-ba8683d76e51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:12:27.837031 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-e091b5aa-fac9-4eb9-86e7-ba8683d76e51 None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:12:27.838190 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-e091b5aa-fac9-4eb9-86e7-ba8683d76e51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:12:27.902258 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 541/1086] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 12:12:27 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 15 12:12:28.171258 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e532484-babe-40ea-b121-573e80ebbdaa None None] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:12:28.268380 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:12:28.273080 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e532484-babe-40ea-b121-573e80ebbdaa None None] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:12:28.306518 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfaf4d21-7bf3-47ce-8ceb-fd4ba7591bc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:12:28.307149 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfaf4d21-7bf3-47ce-8ceb-fd4ba7591bc6 None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:12:28.307429 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfaf4d21-7bf3-47ce-8ceb-fd4ba7591bc6 None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:12:28.307701 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfaf4d21-7bf3-47ce-8ceb-fd4ba7591bc6 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:12:28.317783 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfaf4d21-7bf3-47ce-8ceb-fd4ba7591bc6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 12:12:28.322537 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application [None req-dfaf4d21-7bf3-47ce-8ceb-fd4ba7591bc6 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 15 12:12:28.322537 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:12:28.322537 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:12:28.322537 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:12:28.322537 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:12:28.322537 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:12:28.322537 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:12:28.322537 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:12:28.322537 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:12:28.322537 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:12:28.322537 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:12:28.322537 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:12:28.322537 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 12:12:28.322537 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 12:12:28.322537 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 12:12:28.322537 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 12:12:28.323629 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 12:12:28.323629 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 12:12:28.323629 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 12:12:28.323629 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 12:12:28.323629 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:12:28.323629 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:12:28.323629 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:12:28.323629 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:12:28.323629 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:12:28.323629 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:12:28.323629 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:12:28.323629 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application with Lock( Sep 15 12:12:28.323629 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:12:28.323629 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:12:28.323629 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:12:28.323629 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:12:28.324489 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:12:28.324489 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:12:28.324489 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:12:28.324489 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:12:28.324489 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 12:12:28.324489 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 12:12:28.324489 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 12:12:28.324489 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 12:12:28.324489 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 12:12:28.324489 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 12:12:28.324489 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Sep 15 12:12:28.324489 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application Sep 15 12:12:28.324489 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 546/1087] 10.222.0.15 () {56 vars in 1120 bytes} [Thu Sep 15 12:12:28 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:12:28.330144 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e091b5aa-fac9-4eb9-86e7-ba8683d76e51 None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:12:28.444652 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e091b5aa-fac9-4eb9-86e7-ba8683d76e51 None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:12:28.452601 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:12:28.480552 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9827c3a7-5884-4c8d-b4df-df89e539f881 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:12:28.481194 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9827c3a7-5884-4c8d-b4df-df89e539f881 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:12:28.481507 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9827c3a7-5884-4c8d-b4df-df89e539f881 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:12:28.481740 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9827c3a7-5884-4c8d-b4df-df89e539f881 None admin] PATH_INFO: `/v3/roles` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:12:28.482400 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9827c3a7-5884-4c8d-b4df-df89e539f881 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:12:28.482700 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9827c3a7-5884-4c8d-b4df-df89e539f881 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:12:28.484065 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9827c3a7-5884-4c8d-b4df-df89e539f881 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:12:28.498146 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 542/1088] 10.222.0.15 () {56 vars in 1115 bytes} [Thu Sep 15 12:12:28 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:12:28.499080 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfaf4d21-7bf3-47ce-8ceb-fd4ba7591bc6 None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:12:28.618946 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfaf4d21-7bf3-47ce-8ceb-fd4ba7591bc6 None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:12:28.654841 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4165d6-5616-4101-a9c8-b8a6f66b8ff7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:12:28.655474 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4165d6-5616-4101-a9c8-b8a6f66b8ff7 None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:12:28.655751 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4165d6-5616-4101-a9c8-b8a6f66b8ff7 None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:12:28.656021 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4165d6-5616-4101-a9c8-b8a6f66b8ff7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:12:28.666723 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4165d6-5616-4101-a9c8-b8a6f66b8ff7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:12:28.668164 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4165d6-5616-4101-a9c8-b8a6f66b8ff7 None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:12:28.682433 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application [None req-bb4165d6-5616-4101-a9c8-b8a6f66b8ff7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 12:12:28.682433 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:12:28.682433 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:12:28.682433 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:12:28.682433 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:12:28.682433 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:12:28.682433 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:12:28.682433 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:12:28.682433 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:12:28.682433 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:12:28.682433 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:12:28.682433 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:12:28.682433 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 12:12:28.682433 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 12:12:28.682433 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 12:12:28.682433 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 12:12:28.683609 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:12:28.683609 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:12:28.683609 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:12:28.683609 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:12:28.683609 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:12:28.683609 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:12:28.683609 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:12:28.683609 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application with Lock( Sep 15 12:12:28.683609 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:12:28.683609 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:12:28.683609 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:12:28.683609 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:12:28.683609 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:12:28.683609 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:12:28.683609 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:12:28.683609 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:12:28.683609 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 12:12:28.684600 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 12:12:28.684600 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 12:12:28.684600 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application Sep 15 12:12:28.684600 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 547/1089] 10.222.0.15 () {56 vars in 1108 bytes} [Thu Sep 15 12:12:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:12:28.690819 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9827c3a7-5884-4c8d-b4df-df89e539f881 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:12:28.698586 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b513b3-eb55-4119-9dc4-52163bcc5f6b None None] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:12:28.698797 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b513b3-eb55-4119-9dc4-52163bcc5f6b None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:12:28.699063 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b513b3-eb55-4119-9dc4-52163bcc5f6b None None] PATH_INFO: `/` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:12:28.700360 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 548/1090] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 12:12:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 15 12:12:28.705649 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cf1429-a234-4070-8437-24c2335cc388 None None] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:12:28.705922 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cf1429-a234-4070-8437-24c2335cc388 None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:12:28.706187 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cf1429-a234-4070-8437-24c2335cc388 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:12:28.724044 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.auth.core [None req-78cf1429-a234-4070-8437-24c2335cc388 None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81449) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:12:28.725278 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-78cf1429-a234-4070-8437-24c2335cc388 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:12:28.732679 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:12:28.793286 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 549/1091] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 12:12:28 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 15 12:12:28.795808 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9827c3a7-5884-4c8d-b4df-df89e539f881 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:12:28.828012 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70036c93-80ee-44b2-b750-eed1ab57b001 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:12:28.828622 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70036c93-80ee-44b2-b750-eed1ab57b001 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:12:28.828937 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70036c93-80ee-44b2-b750-eed1ab57b001 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:12:28.829206 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70036c93-80ee-44b2-b750-eed1ab57b001 None admin] PATH_INFO: `/v3/domains` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:12:28.829858 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70036c93-80ee-44b2-b750-eed1ab57b001 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:12:28.830691 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70036c93-80ee-44b2-b750-eed1ab57b001 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:12:28.832028 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70036c93-80ee-44b2-b750-eed1ab57b001 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:12:28.843297 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 543/1092] 10.222.0.15 () {56 vars in 1109 bytes} [Thu Sep 15 12:12:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 12:12:28.849604 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78cf1429-a234-4070-8437-24c2335cc388 None None] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:12:28.945867 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78cf1429-a234-4070-8437-24c2335cc388 None None] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:12:28.981607 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a4f961-6595-4505-ba7e-cff9f923a1e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:12:28.982238 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a4f961-6595-4505-ba7e-cff9f923a1e2 None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:12:28.982516 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a4f961-6595-4505-ba7e-cff9f923a1e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:12:28.982785 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a4f961-6595-4505-ba7e-cff9f923a1e2 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:12:28.989904 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a4f961-6595-4505-ba7e-cff9f923a1e2 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:12:28.991367 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a4f961-6595-4505-ba7e-cff9f923a1e2 None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:12:29.003241 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application [None req-59a4f961-6595-4505-ba7e-cff9f923a1e2 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Sep 15 12:12:29.003241 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:12:29.003241 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:12:29.003241 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:12:29.003241 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:12:29.003241 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:12:29.003241 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:12:29.003241 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:12:29.003241 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:12:29.003241 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:12:29.003241 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:12:29.003241 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:12:29.003241 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 15 12:12:29.003241 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 15 12:12:29.003241 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 15 12:12:29.003241 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 15 12:12:29.004377 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:12:29.004377 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:12:29.004377 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 15 12:12:29.004377 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 12:12:29.004377 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 15 12:12:29.004377 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 12:12:29.004377 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:12:29.004377 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:12:29.004377 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:12:29.004377 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:12:29.004377 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:12:29.004377 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application with Lock( Sep 15 12:12:29.004377 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:12:29.004377 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:12:29.004377 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:12:29.004377 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:12:29.004377 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:12:29.004377 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:12:29.005396 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:12:29.005396 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:12:29.005396 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 15 12:12:29.005396 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 15 12:12:29.005396 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 15 12:12:29.005396 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 15 12:12:29.005396 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 15 12:12:29.005396 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 15 12:12:29.005396 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Sep 15 12:12:29.005396 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application Sep 15 12:12:29.005396 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 550/1093] 10.222.0.15 () {56 vars in 1093 bytes} [Thu Sep 15 12:12:28 2022] GET /identity/v3/users/nova => generated 82 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 15 12:12:29.010222 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70036c93-80ee-44b2-b750-eed1ab57b001 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:12:29.037708 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:12:29.079639 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a4f961-6595-4505-ba7e-cff9f923a1e2 None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:12:29.083023 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:12:29.107260 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70036c93-80ee-44b2-b750-eed1ab57b001 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:12:29.141763 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2fd95b-f72b-457f-9cf7-224a826a7549 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:12:29.142370 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2fd95b-f72b-457f-9cf7-224a826a7549 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:12:29.143228 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2fd95b-f72b-457f-9cf7-224a826a7549 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:12:29.143500 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2fd95b-f72b-457f-9cf7-224a826a7549 None admin] PATH_INFO: `/v3/users` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:12:29.144205 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2fd95b-f72b-457f-9cf7-224a826a7549 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:12:29.144503 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2fd95b-f72b-457f-9cf7-224a826a7549 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:12:29.145940 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2fd95b-f72b-457f-9cf7-224a826a7549 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:12:29.162671 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 544/1094] 10.222.0.15 () {56 vars in 1133 bytes} [Thu Sep 15 12:12:29 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:12:29.168555 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2fd95b-f72b-457f-9cf7-224a826a7549 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:12:29.180990 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a4f961-6595-4505-ba7e-cff9f923a1e2 None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:12:29.214741 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0746306-3c62-4857-97e0-43eaa8dd295f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:12:29.215361 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0746306-3c62-4857-97e0-43eaa8dd295f None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:12:29.215638 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0746306-3c62-4857-97e0-43eaa8dd295f None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:12:29.215905 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0746306-3c62-4857-97e0-43eaa8dd295f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:12:29.227606 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0746306-3c62-4857-97e0-43eaa8dd295f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 12:12:29.232497 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application [None req-c0746306-3c62-4857-97e0-43eaa8dd295f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 15 12:12:29.232497 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:12:29.232497 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:12:29.232497 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:12:29.232497 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:12:29.232497 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:12:29.232497 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:12:29.232497 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:12:29.232497 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:12:29.232497 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:12:29.232497 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:12:29.232497 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:12:29.232497 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 12:12:29.232497 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 12:12:29.232497 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 12:12:29.232497 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 12:12:29.233683 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 12:12:29.233683 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 12:12:29.233683 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 12:12:29.233683 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 12:12:29.233683 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:12:29.233683 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:12:29.233683 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:12:29.233683 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:12:29.233683 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:12:29.233683 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:12:29.233683 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:12:29.233683 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application with Lock( Sep 15 12:12:29.233683 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:12:29.233683 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:12:29.233683 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:12:29.233683 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:12:29.235488 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:12:29.235488 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:12:29.235488 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:12:29.235488 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:12:29.235488 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 12:12:29.235488 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 12:12:29.235488 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 12:12:29.235488 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 12:12:29.235488 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 12:12:29.235488 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 12:12:29.235488 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 15 12:12:29.235488 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application Sep 15 12:12:29.235488 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 551/1095] 10.222.0.15 () {56 vars in 1096 bytes} [Thu Sep 15 12:12:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 15 12:12:29.240474 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0746306-3c62-4857-97e0-43eaa8dd295f None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:12:29.256659 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:12:29.272984 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2fd95b-f72b-457f-9cf7-224a826a7549 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:12:29.304295 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:12:29.304968 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a716044-40cb-4d0c-a99a-a73f9f7cd68c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:12:29.305522 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a716044-40cb-4d0c-a99a-a73f9f7cd68c None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:12:29.305798 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a716044-40cb-4d0c-a99a-a73f9f7cd68c None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:12:29.306062 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a716044-40cb-4d0c-a99a-a73f9f7cd68c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:12:29.343313 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0746306-3c62-4857-97e0-43eaa8dd295f None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:12:29.371511 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a716044-40cb-4d0c-a99a-a73f9f7cd68c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:12:29.372964 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a716044-40cb-4d0c-a99a-a73f9f7cd68c None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:12:29.379158 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b8d0a5-4fac-4484-91f7-2ccc130b222b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:12:29.379758 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b8d0a5-4fac-4484-91f7-2ccc130b222b None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:12:29.380035 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b8d0a5-4fac-4484-91f7-2ccc130b222b None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:12:29.380305 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b8d0a5-4fac-4484-91f7-2ccc130b222b None admin] PATH_INFO: `/v3/roles` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:12:29.380989 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b8d0a5-4fac-4484-91f7-2ccc130b222b None admin] RBAC: Adding query filter params (name=admin) {{(pid=81449) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:12:29.381286 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b8d0a5-4fac-4484-91f7-2ccc130b222b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:12:29.382630 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b8d0a5-4fac-4484-91f7-2ccc130b222b None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:12:29.396612 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 552/1096] 10.222.0.15 () {56 vars in 1099 bytes} [Thu Sep 15 12:12:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 12:12:29.397645 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b8d0a5-4fac-4484-91f7-2ccc130b222b None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:12:29.456087 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:12:29.471354 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 545/1097] 10.222.0.15 () {58 vars in 1282 bytes} [Thu Sep 15 12:12:29 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 15 12:12:29.477137 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a716044-40cb-4d0c-a99a-a73f9f7cd68c None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:12:29.493759 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b8d0a5-4fac-4484-91f7-2ccc130b222b None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:12:29.517312 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:12:29.528720 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbfbf3f-aa12-49bd-be9b-275b9cb2c5fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:12:29.529387 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbfbf3f-aa12-49bd-be9b-275b9cb2c5fb None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:12:29.529666 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbfbf3f-aa12-49bd-be9b-275b9cb2c5fb None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:12:29.529939 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbfbf3f-aa12-49bd-be9b-275b9cb2c5fb None admin] PATH_INFO: `/v3/users/2dbccf3842964c0ea8543e21f82cc76d` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:12:29.531068 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bbfbf3f-aa12-49bd-be9b-275b9cb2c5fb None admin] RBAC: Authorizing `identity:get_user(user_id=2dbccf3842964c0ea8543e21f82cc76d)` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:12:29.532486 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bbfbf3f-aa12-49bd-be9b-275b9cb2c5fb None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:12:29.534108 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 553/1098] 10.222.0.15 () {56 vars in 1177 bytes} [Thu Sep 15 12:12:29 2022] GET /identity/v3/users/2dbccf3842964c0ea8543e21f82cc76d => generated 285 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 12:12:29.539876 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbfbf3f-aa12-49bd-be9b-275b9cb2c5fb None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:12:29.576397 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a716044-40cb-4d0c-a99a-a73f9f7cd68c None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:12:29.609124 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f6257f-d3c4-4f98-9cc8-63c8d46a7bfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:12:29.609909 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f6257f-d3c4-4f98-9cc8-63c8d46a7bfb None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:12:29.610157 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f6257f-d3c4-4f98-9cc8-63c8d46a7bfb None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:12:29.610957 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f6257f-d3c4-4f98-9cc8-63c8d46a7bfb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:12:29.621762 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f6257f-d3c4-4f98-9cc8-63c8d46a7bfb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:12:29.623402 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f6257f-d3c4-4f98-9cc8-63c8d46a7bfb None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:12:29.637759 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbfbf3f-aa12-49bd-be9b-275b9cb2c5fb None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:12:29.639457 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application [None req-54f6257f-d3c4-4f98-9cc8-63c8d46a7bfb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 12:12:29.639457 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:12:29.639457 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:12:29.639457 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:12:29.639457 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:12:29.639457 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:12:29.639457 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:12:29.639457 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:12:29.639457 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:12:29.639457 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:12:29.639457 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:12:29.639457 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:12:29.639457 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 12:12:29.639457 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 12:12:29.639457 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 12:12:29.639457 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 12:12:29.640604 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:12:29.640604 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:12:29.640604 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:12:29.640604 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:12:29.640604 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:12:29.640604 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:12:29.640604 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:12:29.640604 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application with Lock( Sep 15 12:12:29.640604 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:12:29.640604 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:12:29.640604 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:12:29.640604 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:12:29.640604 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:12:29.640604 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:12:29.640604 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:12:29.640604 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:12:29.640604 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 12:12:29.642364 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 12:12:29.642364 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 12:12:29.642364 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 15 12:12:29.642364 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 546/1099] 10.222.0.15 () {56 vars in 1108 bytes} [Thu Sep 15 12:12:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 15 12:12:29.646862 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f6257f-d3c4-4f98-9cc8-63c8d46a7bfb None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:12:29.676698 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec09ef91-688e-4b66-b2bc-aa8a458dfbea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:12:29.677403 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec09ef91-688e-4b66-b2bc-aa8a458dfbea None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:12:29.677685 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec09ef91-688e-4b66-b2bc-aa8a458dfbea None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:12:29.677953 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec09ef91-688e-4b66-b2bc-aa8a458dfbea None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:12:29.687466 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:12:29.734927 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:12:29.744647 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec09ef91-688e-4b66-b2bc-aa8a458dfbea None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:12:29.746115 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec09ef91-688e-4b66-b2bc-aa8a458dfbea None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:12:29.748531 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f6257f-d3c4-4f98-9cc8-63c8d46a7bfb None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:12:29.781148 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e2d582-e420-4442-9e2f-6e8d929abef2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:12:29.781751 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e2d582-e420-4442-9e2f-6e8d929abef2 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:12:29.782026 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e2d582-e420-4442-9e2f-6e8d929abef2 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:12:29.782291 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e2d582-e420-4442-9e2f-6e8d929abef2 None admin] PATH_INFO: `/v3/domains` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:12:29.782919 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e2d582-e420-4442-9e2f-6e8d929abef2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:12:29.783750 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e2d582-e420-4442-9e2f-6e8d929abef2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:12:29.785110 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e2d582-e420-4442-9e2f-6e8d929abef2 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:12:29.796583 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 547/1100] 10.222.0.15 () {56 vars in 1109 bytes} [Thu Sep 15 12:12:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 12:12:29.802809 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e2d582-e420-4442-9e2f-6e8d929abef2 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:12:29.841440 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 554/1101] 10.222.0.15 () {58 vars in 1282 bytes} [Thu Sep 15 12:12:29 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Sep 15 12:12:29.847817 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec09ef91-688e-4b66-b2bc-aa8a458dfbea None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:12:29.896967 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:12:29.900904 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e2d582-e420-4442-9e2f-6e8d929abef2 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:12:29.934888 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:12:29.935408 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d99976-f812-4cb4-8480-59f1b7f43396 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:12:29.936085 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d99976-f812-4cb4-8480-59f1b7f43396 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:12:29.936391 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d99976-f812-4cb4-8480-59f1b7f43396 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:12:29.936730 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d99976-f812-4cb4-8480-59f1b7f43396 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:12:29.943238 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d99976-f812-4cb4-8480-59f1b7f43396 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:12:29.943964 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec09ef91-688e-4b66-b2bc-aa8a458dfbea None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:12:29.944645 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d99976-f812-4cb4-8480-59f1b7f43396 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:12:29.955506 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application [None req-96d99976-f812-4cb4-8480-59f1b7f43396 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 12:12:29.955506 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:12:29.955506 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:12:29.955506 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:12:29.955506 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:12:29.955506 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:12:29.955506 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:12:29.955506 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:12:29.955506 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:12:29.955506 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:12:29.955506 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:12:29.955506 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:12:29.955506 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 15 12:12:29.955506 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 15 12:12:29.955506 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 15 12:12:29.956445 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 15 12:12:29.956445 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:12:29.956445 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:12:29.956445 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:12:29.956445 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:12:29.956445 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:12:29.956445 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:12:29.956445 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:12:29.956445 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application with Lock( Sep 15 12:12:29.956445 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:12:29.956445 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:12:29.956445 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:12:29.956445 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:12:29.956445 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:12:29.956445 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:12:29.956445 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:12:29.956445 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:12:29.957405 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 15 12:12:29.957405 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 15 12:12:29.957405 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 15 12:12:29.957405 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 15 12:12:29.957405 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 15 12:12:29.957405 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 15 12:12:29.957405 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 12:12:29.957405 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 15 12:12:29.957405 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 548/1102] 10.222.0.15 () {56 vars in 1111 bytes} [Thu Sep 15 12:12:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 15 12:12:29.962775 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d99976-f812-4cb4-8480-59f1b7f43396 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:12:29.981945 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42858974-6ae7-4e5d-9007-bc979e4850a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:12:29.982579 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42858974-6ae7-4e5d-9007-bc979e4850a2 None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:12:29.982856 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42858974-6ae7-4e5d-9007-bc979e4850a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:12:29.983128 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42858974-6ae7-4e5d-9007-bc979e4850a2 None admin] PATH_INFO: `/v3/projects/b7fb192a30504033bcad23ae818e1e8c` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:12:29.984233 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42858974-6ae7-4e5d-9007-bc979e4850a2 None admin] RBAC: Authorizing `identity:get_project(project_id=b7fb192a30504033bcad23ae818e1e8c)` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:12:29.985679 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42858974-6ae7-4e5d-9007-bc979e4850a2 None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:12:29.987920 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 555/1103] 10.222.0.15 () {56 vars in 1186 bytes} [Thu Sep 15 12:12:29 2022] GET /identity/v3/projects/b7fb192a30504033bcad23ae818e1e8c => generated 311 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 12:12:29.993908 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42858974-6ae7-4e5d-9007-bc979e4850a2 None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:12:30.061888 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d99976-f812-4cb4-8480-59f1b7f43396 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:12:30.090943 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42858974-6ae7-4e5d-9007-bc979e4850a2 None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:12:30.093977 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98585f40-0020-41ad-9a4f-7aab59c7feea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:12:30.094581 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98585f40-0020-41ad-9a4f-7aab59c7feea None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:12:30.094856 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98585f40-0020-41ad-9a4f-7aab59c7feea None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:12:30.095121 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98585f40-0020-41ad-9a4f-7aab59c7feea None admin] PATH_INFO: `/v3/projects` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:12:30.096356 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98585f40-0020-41ad-9a4f-7aab59c7feea None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:12:30.096655 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98585f40-0020-41ad-9a4f-7aab59c7feea None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:12:30.098130 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98585f40-0020-41ad-9a4f-7aab59c7feea None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:12:30.110225 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 549/1104] 10.222.0.15 () {56 vars in 1148 bytes} [Thu Sep 15 12:12:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 12:12:30.116876 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98585f40-0020-41ad-9a4f-7aab59c7feea None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:12:30.124590 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eabd027-c07e-420b-9f10-aa6a58c2bde6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:12:30.125217 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eabd027-c07e-420b-9f10-aa6a58c2bde6 None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:12:30.125491 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eabd027-c07e-420b-9f10-aa6a58c2bde6 None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:12:30.125756 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eabd027-c07e-420b-9f10-aa6a58c2bde6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:12:30.126463 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eabd027-c07e-420b-9f10-aa6a58c2bde6 None admin] RBAC: Adding query filter params (role.id=9c769b5c82884766b2c510b1f0f1364c, scope.project.id=b7fb192a30504033bcad23ae818e1e8c, user.id=2dbccf3842964c0ea8543e21f82cc76d) {{(pid=81449) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:12:30.126780 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eabd027-c07e-420b-9f10-aa6a58c2bde6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:12:30.128189 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eabd027-c07e-420b-9f10-aa6a58c2bde6 None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:12:30.133665 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 556/1105] 10.222.0.15 () {56 vars in 1374 bytes} [Thu Sep 15 12:12:29 2022] GET /identity/v3/role_assignments?user.id=2dbccf3842964c0ea8543e21f82cc76d&scope.project.id=b7fb192a30504033bcad23ae818e1e8c&role.id=9c769b5c82884766b2c510b1f0f1364c => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 12:12:30.134616 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:12:30.220278 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98585f40-0020-41ad-9a4f-7aab59c7feea None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:12:30.255500 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe54390-8c36-4cd5-a1d3-487f8a36b6c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:12:30.256134 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe54390-8c36-4cd5-a1d3-487f8a36b6c3 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:12:30.256410 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe54390-8c36-4cd5-a1d3-487f8a36b6c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:12:30.256676 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe54390-8c36-4cd5-a1d3-487f8a36b6c3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:12:30.257441 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe54390-8c36-4cd5-a1d3-487f8a36b6c3 None admin] RBAC: Adding query filter params (role.id=c0320da65e0b4cdd8924c20a99b978af, scope.project.id=540a32ef88764e8c8d9201ae22f6ddaf, user.id=eface89e261840a2abb4206be26455c7) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:12:30.257746 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe54390-8c36-4cd5-a1d3-487f8a36b6c3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:12:30.259199 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe54390-8c36-4cd5-a1d3-487f8a36b6c3 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:12:30.276785 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 550/1106] 10.222.0.15 () {56 vars in 1374 bytes} [Thu Sep 15 12:12:30 2022] GET /identity/v3/role_assignments?user.id=eface89e261840a2abb4206be26455c7&scope.project.id=540a32ef88764e8c8d9201ae22f6ddaf&role.id=c0320da65e0b4cdd8924c20a99b978af => generated 608 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 12:12:32.302185 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c51e7b-848e-4bcf-bf4e-d37ee7280c73 None None] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:12:32.302681 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c51e7b-848e-4bcf-bf4e-d37ee7280c73 None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:12:32.302681 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c51e7b-848e-4bcf-bf4e-d37ee7280c73 None None] PATH_INFO: `/` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:12:32.303974 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 557/1107] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 12:12:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:12:32.309382 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1860db2f-3a4c-4052-a627-37bd96f667df None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:12:32.309803 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1860db2f-3a4c-4052-a627-37bd96f667df None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:12:32.309922 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1860db2f-3a4c-4052-a627-37bd96f667df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:12:32.328913 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-1860db2f-3a4c-4052-a627-37bd96f667df None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:12:32.330668 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-1860db2f-3a4c-4052-a627-37bd96f667df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:12:32.393658 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 551/1108] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 12:12:32 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 15 12:12:32.666509 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c51e7b-848e-4bcf-bf4e-d37ee7280c73 None None] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:12:32.690517 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:12:32.766518 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c51e7b-848e-4bcf-bf4e-d37ee7280c73 None None] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:12:32.799296 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72ee111-a3fb-42f1-a397-9eb36b2ee60d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:12:32.799918 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72ee111-a3fb-42f1-a397-9eb36b2ee60d None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:12:32.800193 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72ee111-a3fb-42f1-a397-9eb36b2ee60d None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:12:32.800462 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72ee111-a3fb-42f1-a397-9eb36b2ee60d None admin] PATH_INFO: `/v3/domains/default` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:12:32.801589 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72ee111-a3fb-42f1-a397-9eb36b2ee60d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:12:32.803008 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72ee111-a3fb-42f1-a397-9eb36b2ee60d None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:12:32.804543 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 558/1109] 10.222.0.15 () {56 vars in 1108 bytes} [Thu Sep 15 12:12:32 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:12:32.810314 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1860db2f-3a4c-4052-a627-37bd96f667df None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:12:32.824371 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:12:32.905874 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1860db2f-3a4c-4052-a627-37bd96f667df None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:12:32.937892 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3729d1f-9279-49b0-93ac-5d5b0f7ce274 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:12:32.938492 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3729d1f-9279-49b0-93ac-5d5b0f7ce274 None admin] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:12:32.938768 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3729d1f-9279-49b0-93ac-5d5b0f7ce274 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:12:32.939033 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3729d1f-9279-49b0-93ac-5d5b0f7ce274 None admin] PATH_INFO: `/v3/users` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:12:32.943514 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3729d1f-9279-49b0-93ac-5d5b0f7ce274 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:12:32.945009 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3729d1f-9279-49b0-93ac-5d5b0f7ce274 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:12:32.967932 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 552/1110] 10.222.0.15 () {60 vars in 1132 bytes} [Thu Sep 15 12:12:32 2022] POST /identity/v3/users => generated 286 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 12:12:35.124453 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d657750-7bbe-41e2-810b-10cb97093c39 None None] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:12:35.125163 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d657750-7bbe-41e2-810b-10cb97093c39 None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:12:35.125163 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d657750-7bbe-41e2-810b-10cb97093c39 None None] PATH_INFO: `/` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:12:35.126394 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 559/1111] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 12:12:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:12:35.132102 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b166af-2c15-49a3-bbf2-c1fe3ac7f1ca None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:12:35.132484 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b166af-2c15-49a3-bbf2-c1fe3ac7f1ca None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:12:35.132645 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b166af-2c15-49a3-bbf2-c1fe3ac7f1ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:12:35.150784 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-37b166af-2c15-49a3-bbf2-c1fe3ac7f1ca None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:12:35.151958 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-37b166af-2c15-49a3-bbf2-c1fe3ac7f1ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:12:35.210050 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:12:35.216189 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 553/1112] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 12:12:35 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 15 12:12:35.482877 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d657750-7bbe-41e2-810b-10cb97093c39 None None] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:12:35.580840 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:12:35.594435 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d657750-7bbe-41e2-810b-10cb97093c39 None None] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:12:35.633218 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654b748b-eedb-49ff-b9e5-1f6d1973c12a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:12:35.633838 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654b748b-eedb-49ff-b9e5-1f6d1973c12a None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:12:35.634112 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654b748b-eedb-49ff-b9e5-1f6d1973c12a None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:12:35.634380 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654b748b-eedb-49ff-b9e5-1f6d1973c12a None admin] PATH_INFO: `/v3/roles/1081b93ddfd547649ba8582f7a6e719e` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:12:35.635726 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-654b748b-eedb-49ff-b9e5-1f6d1973c12a None admin] RBAC: Authorizing `identity:get_role(role_id=1081b93ddfd547649ba8582f7a6e719e)` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:12:35.637099 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-654b748b-eedb-49ff-b9e5-1f6d1973c12a None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:12:35.638380 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 560/1113] 10.222.0.15 () {56 vars in 1177 bytes} [Thu Sep 15 12:12:35 2022] GET /identity/v3/roles/1081b93ddfd547649ba8582f7a6e719e => generated 223 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:12:35.644173 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b166af-2c15-49a3-bbf2-c1fe3ac7f1ca None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:12:35.738550 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b166af-2c15-49a3-bbf2-c1fe3ac7f1ca None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:12:35.771066 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4571746-2b8a-4fb7-99f8-2c228166381f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:12:35.771684 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4571746-2b8a-4fb7-99f8-2c228166381f None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:12:35.771960 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4571746-2b8a-4fb7-99f8-2c228166381f None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:12:35.772228 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4571746-2b8a-4fb7-99f8-2c228166381f None admin] PATH_INFO: `/v3/users/2c8fdbd8913c4d109da4d725e9e626a6` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:12:35.784861 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4571746-2b8a-4fb7-99f8-2c228166381f None admin] RBAC: Authorizing `identity:get_user(user_id=2c8fdbd8913c4d109da4d725e9e626a6)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:12:35.786273 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4571746-2b8a-4fb7-99f8-2c228166381f None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:12:35.787834 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 554/1114] 10.222.0.15 () {56 vars in 1177 bytes} [Thu Sep 15 12:12:35 2022] GET /identity/v3/users/2c8fdbd8913c4d109da4d725e9e626a6 => generated 286 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:12:35.793532 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654b748b-eedb-49ff-b9e5-1f6d1973c12a None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:12:35.891823 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654b748b-eedb-49ff-b9e5-1f6d1973c12a None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:12:35.924598 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb373d48-b4f2-4413-9b39-8e3a9b2b41d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:12:35.925230 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb373d48-b4f2-4413-9b39-8e3a9b2b41d4 None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:12:35.925509 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb373d48-b4f2-4413-9b39-8e3a9b2b41d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:12:35.925778 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb373d48-b4f2-4413-9b39-8e3a9b2b41d4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:12:35.952090 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:12:35.991018 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb373d48-b4f2-4413-9b39-8e3a9b2b41d4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:12:35.992352 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb373d48-b4f2-4413-9b39-8e3a9b2b41d4 None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:12:36.086185 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 561/1115] 10.222.0.15 () {58 vars in 1282 bytes} [Thu Sep 15 12:12:35 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 15 12:12:36.092077 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4571746-2b8a-4fb7-99f8-2c228166381f None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:12:36.146317 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:12:36.187987 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4571746-2b8a-4fb7-99f8-2c228166381f None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:12:36.220243 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b434a590-0ae3-4772-be48-aae4b2292180 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:12:36.220900 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b434a590-0ae3-4772-be48-aae4b2292180 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:12:36.221183 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b434a590-0ae3-4772-be48-aae4b2292180 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:12:36.221456 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b434a590-0ae3-4772-be48-aae4b2292180 None admin] PATH_INFO: `/v3/projects/b7fb192a30504033bcad23ae818e1e8c` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:12:36.222550 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b434a590-0ae3-4772-be48-aae4b2292180 None admin] RBAC: Authorizing `identity:get_project(project_id=b7fb192a30504033bcad23ae818e1e8c)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:12:36.223960 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b434a590-0ae3-4772-be48-aae4b2292180 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:12:36.225565 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 555/1116] 10.222.0.15 () {56 vars in 1186 bytes} [Thu Sep 15 12:12:36 2022] GET /identity/v3/projects/b7fb192a30504033bcad23ae818e1e8c => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:12:36.231785 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb373d48-b4f2-4413-9b39-8e3a9b2b41d4 None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:12:36.288242 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:12:36.328559 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb373d48-b4f2-4413-9b39-8e3a9b2b41d4 None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:12:36.361653 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36026267-2bfb-4b70-830a-ecb98d68a087 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:12:36.362246 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36026267-2bfb-4b70-830a-ecb98d68a087 None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:12:36.362526 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36026267-2bfb-4b70-830a-ecb98d68a087 None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:12:36.362796 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36026267-2bfb-4b70-830a-ecb98d68a087 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:12:36.363503 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36026267-2bfb-4b70-830a-ecb98d68a087 None admin] RBAC: Adding query filter params (role.id=1081b93ddfd547649ba8582f7a6e719e, scope.project.id=b7fb192a30504033bcad23ae818e1e8c, user.id=2c8fdbd8913c4d109da4d725e9e626a6) {{(pid=81449) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:12:36.363810 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36026267-2bfb-4b70-830a-ecb98d68a087 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:12:36.365256 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36026267-2bfb-4b70-830a-ecb98d68a087 None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:12:36.370422 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 562/1117] 10.222.0.15 () {56 vars in 1374 bytes} [Thu Sep 15 12:12:36 2022] GET /identity/v3/role_assignments?user.id=2c8fdbd8913c4d109da4d725e9e626a6&scope.project.id=b7fb192a30504033bcad23ae818e1e8c&role.id=1081b93ddfd547649ba8582f7a6e719e => generated 259 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:12:38.498201 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61158339-b9f4-4fcf-87ca-85f01585909e None None] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:12:38.498721 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61158339-b9f4-4fcf-87ca-85f01585909e None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:12:38.498810 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61158339-b9f4-4fcf-87ca-85f01585909e None None] PATH_INFO: `/` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:12:38.500079 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 556/1118] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 12:12:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:12:38.505397 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ec1e2f-de6c-48e7-ae6c-add0ff0c1b34 None None] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:12:38.505812 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ec1e2f-de6c-48e7-ae6c-add0ff0c1b34 None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:12:38.506027 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ec1e2f-de6c-48e7-ae6c-add0ff0c1b34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:12:38.523998 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.auth.core [None req-24ec1e2f-de6c-48e7-ae6c-add0ff0c1b34 None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81449) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:12:38.525190 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-24ec1e2f-de6c-48e7-ae6c-add0ff0c1b34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:12:38.594393 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 563/1119] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 12:12:38 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 15 12:12:38.866996 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61158339-b9f4-4fcf-87ca-85f01585909e None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:12:38.965321 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61158339-b9f4-4fcf-87ca-85f01585909e None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:12:38.990674 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:12:39.002463 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479f2630-c535-4425-a094-7cdedfa05e43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:12:39.003105 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479f2630-c535-4425-a094-7cdedfa05e43 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:12:39.003378 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479f2630-c535-4425-a094-7cdedfa05e43 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:12:39.003645 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479f2630-c535-4425-a094-7cdedfa05e43 None admin] PATH_INFO: `/v3/roles/1081b93ddfd547649ba8582f7a6e719e` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:12:39.005058 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479f2630-c535-4425-a094-7cdedfa05e43 None admin] RBAC: Authorizing `identity:get_role(role_id=1081b93ddfd547649ba8582f7a6e719e)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:12:39.006412 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-479f2630-c535-4425-a094-7cdedfa05e43 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:12:39.007722 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 557/1120] 10.222.0.15 () {56 vars in 1177 bytes} [Thu Sep 15 12:12:38 2022] GET /identity/v3/roles/1081b93ddfd547649ba8582f7a6e719e => generated 223 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:12:39.014489 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ec1e2f-de6c-48e7-ae6c-add0ff0c1b34 None None] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:12:39.067112 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:12:39.109873 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ec1e2f-de6c-48e7-ae6c-add0ff0c1b34 None None] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:12:39.141854 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff401e2-66d6-4e0b-916c-98db57771247 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:12:39.142953 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff401e2-66d6-4e0b-916c-98db57771247 None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:12:39.143229 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff401e2-66d6-4e0b-916c-98db57771247 None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:12:39.143499 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff401e2-66d6-4e0b-916c-98db57771247 None admin] PATH_INFO: `/v3/users/2c8fdbd8913c4d109da4d725e9e626a6` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:12:39.144600 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff401e2-66d6-4e0b-916c-98db57771247 None admin] RBAC: Authorizing `identity:get_user(user_id=2c8fdbd8913c4d109da4d725e9e626a6)` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:12:39.146040 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff401e2-66d6-4e0b-916c-98db57771247 None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:12:39.147553 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 564/1121] 10.222.0.15 () {56 vars in 1177 bytes} [Thu Sep 15 12:12:39 2022] GET /identity/v3/users/2c8fdbd8913c4d109da4d725e9e626a6 => generated 286 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:12:39.153393 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479f2630-c535-4425-a094-7cdedfa05e43 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:12:39.247727 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479f2630-c535-4425-a094-7cdedfa05e43 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:12:39.279794 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91409aca-ab63-4b91-8a3b-eadb21815237 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:12:39.280400 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91409aca-ab63-4b91-8a3b-eadb21815237 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:12:39.280675 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91409aca-ab63-4b91-8a3b-eadb21815237 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:12:39.280958 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91409aca-ab63-4b91-8a3b-eadb21815237 None admin] PATH_INFO: `/v3/projects/b7fb192a30504033bcad23ae818e1e8c` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:12:39.282038 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91409aca-ab63-4b91-8a3b-eadb21815237 None admin] RBAC: Authorizing `identity:get_project(project_id=b7fb192a30504033bcad23ae818e1e8c)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:12:39.283443 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91409aca-ab63-4b91-8a3b-eadb21815237 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:12:39.285039 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 558/1122] 10.222.0.15 () {56 vars in 1186 bytes} [Thu Sep 15 12:12:39 2022] GET /identity/v3/projects/b7fb192a30504033bcad23ae818e1e8c => generated 311 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:12:39.290675 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff401e2-66d6-4e0b-916c-98db57771247 None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:12:39.385830 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff401e2-66d6-4e0b-916c-98db57771247 None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:12:39.407356 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:12:39.419200 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290ba309-8938-4620-b5af-c35a75047646 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:12:39.419845 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290ba309-8938-4620-b5af-c35a75047646 None admin] REQUEST_METHOD: `PUT` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:12:39.420124 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290ba309-8938-4620-b5af-c35a75047646 None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:12:39.420390 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290ba309-8938-4620-b5af-c35a75047646 None admin] PATH_INFO: `/v3/projects/b7fb192a30504033bcad23ae818e1e8c/users/2c8fdbd8913c4d109da4d725e9e626a6/roles/1081b93ddfd547649ba8582f7a6e719e` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:12:39.422486 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-290ba309-8938-4620-b5af-c35a75047646 None admin] RBAC: Authorizing `identity:create_grant(project_id=b7fb192a30504033bcad23ae818e1e8c, user_id=2c8fdbd8913c4d109da4d725e9e626a6, role_id=1081b93ddfd547649ba8582f7a6e719e)` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:12:39.424426 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-290ba309-8938-4620-b5af-c35a75047646 None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:12:39.436335 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 565/1123] 10.222.0.15 () {58 vars in 1439 bytes} [Thu Sep 15 12:12:39 2022] PUT /identity/v3/projects/b7fb192a30504033bcad23ae818e1e8c/users/2c8fdbd8913c4d109da4d725e9e626a6/roles/1081b93ddfd547649ba8582f7a6e719e => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:12:41.526407 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895433a4-77e5-42c9-810f-8224c0d61e1e None None] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:12:41.526932 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895433a4-77e5-42c9-810f-8224c0d61e1e None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:12:41.526932 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895433a4-77e5-42c9-810f-8224c0d61e1e None None] PATH_INFO: `/` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:12:41.528266 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 559/1124] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 12:12:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:12:41.534012 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65f300b-e1e4-4a7e-8d99-1d6d42f72aa5 None None] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:12:41.534405 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65f300b-e1e4-4a7e-8d99-1d6d42f72aa5 None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:12:41.534555 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65f300b-e1e4-4a7e-8d99-1d6d42f72aa5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:12:41.552158 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.auth.core [None req-d65f300b-e1e4-4a7e-8d99-1d6d42f72aa5 None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81449) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:12:41.553361 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-d65f300b-e1e4-4a7e-8d99-1d6d42f72aa5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:12:41.618374 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 566/1125] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 12:12:41 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 15 12:12:41.884081 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895433a4-77e5-42c9-810f-8224c0d61e1e None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:12:41.942481 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:12:41.985833 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895433a4-77e5-42c9-810f-8224c0d61e1e None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:12:42.018673 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b20b2b6-f3b6-49a5-9d48-812167c76fdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:12:42.019301 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b20b2b6-f3b6-49a5-9d48-812167c76fdd None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:12:42.019587 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b20b2b6-f3b6-49a5-9d48-812167c76fdd None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:12:42.019855 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b20b2b6-f3b6-49a5-9d48-812167c76fdd None admin] PATH_INFO: `/v3/roles/1081b93ddfd547649ba8582f7a6e719e` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:12:42.021247 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b20b2b6-f3b6-49a5-9d48-812167c76fdd None admin] RBAC: Authorizing `identity:get_role(role_id=1081b93ddfd547649ba8582f7a6e719e)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:12:42.022596 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b20b2b6-f3b6-49a5-9d48-812167c76fdd None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:12:42.023893 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 560/1126] 10.222.0.15 () {56 vars in 1177 bytes} [Thu Sep 15 12:12:41 2022] GET /identity/v3/roles/1081b93ddfd547649ba8582f7a6e719e => generated 223 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:12:42.029619 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65f300b-e1e4-4a7e-8d99-1d6d42f72aa5 None None] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:12:42.123273 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65f300b-e1e4-4a7e-8d99-1d6d42f72aa5 None None] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:12:42.136825 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:12:42.158316 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793056a5-5be9-42d1-9f6d-df65f11462eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:12:42.158932 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793056a5-5be9-42d1-9f6d-df65f11462eb None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:12:42.159205 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793056a5-5be9-42d1-9f6d-df65f11462eb None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:12:42.159467 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793056a5-5be9-42d1-9f6d-df65f11462eb None admin] PATH_INFO: `/v3/users/2c8fdbd8913c4d109da4d725e9e626a6` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:12:42.160555 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-793056a5-5be9-42d1-9f6d-df65f11462eb None admin] RBAC: Authorizing `identity:get_user(user_id=2c8fdbd8913c4d109da4d725e9e626a6)` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:12:42.162186 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-793056a5-5be9-42d1-9f6d-df65f11462eb None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:12:42.163893 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 567/1127] 10.222.0.15 () {56 vars in 1177 bytes} [Thu Sep 15 12:12:42 2022] GET /identity/v3/users/2c8fdbd8913c4d109da4d725e9e626a6 => generated 286 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:12:42.169599 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b20b2b6-f3b6-49a5-9d48-812167c76fdd None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:12:42.264209 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b20b2b6-f3b6-49a5-9d48-812167c76fdd None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:12:42.288509 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:12:42.296113 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5ab2a0-4289-4538-b862-d960bef3d87d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:12:42.296703 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5ab2a0-4289-4538-b862-d960bef3d87d None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:12:42.297005 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5ab2a0-4289-4538-b862-d960bef3d87d None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:12:42.297271 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5ab2a0-4289-4538-b862-d960bef3d87d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:12:42.360148 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5ab2a0-4289-4538-b862-d960bef3d87d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:12:42.362128 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5ab2a0-4289-4538-b862-d960bef3d87d None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:12:42.454225 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 561/1128] 10.222.0.15 () {58 vars in 1282 bytes} [Thu Sep 15 12:12:42 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 285 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 15 12:12:42.459935 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793056a5-5be9-42d1-9f6d-df65f11462eb None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:12:42.555320 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793056a5-5be9-42d1-9f6d-df65f11462eb None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:12:42.587054 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed36319a-6d97-4832-a447-894084a9f5a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:12:42.587665 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed36319a-6d97-4832-a447-894084a9f5a4 None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:12:42.587937 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed36319a-6d97-4832-a447-894084a9f5a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:12:42.588200 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed36319a-6d97-4832-a447-894084a9f5a4 None admin] PATH_INFO: `/v3/projects/b7fb192a30504033bcad23ae818e1e8c` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:12:42.589304 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed36319a-6d97-4832-a447-894084a9f5a4 None admin] RBAC: Authorizing `identity:get_project(project_id=b7fb192a30504033bcad23ae818e1e8c)` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:12:42.590706 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed36319a-6d97-4832-a447-894084a9f5a4 None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:12:42.592259 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 568/1129] 10.222.0.15 () {56 vars in 1186 bytes} [Thu Sep 15 12:12:42 2022] GET /identity/v3/projects/b7fb192a30504033bcad23ae818e1e8c => generated 311 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:12:42.598806 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5ab2a0-4289-4538-b862-d960bef3d87d None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:12:42.625704 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:12:42.693566 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5ab2a0-4289-4538-b862-d960bef3d87d None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:12:42.725729 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc85179-5a13-4087-9595-df6c825af7ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:12:42.726259 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc85179-5a13-4087-9595-df6c825af7ab None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:12:42.726533 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc85179-5a13-4087-9595-df6c825af7ab None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:12:42.726800 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc85179-5a13-4087-9595-df6c825af7ab None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:12:42.727507 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc85179-5a13-4087-9595-df6c825af7ab None admin] RBAC: Adding query filter params (role.id=1081b93ddfd547649ba8582f7a6e719e, scope.project.id=b7fb192a30504033bcad23ae818e1e8c, user.id=2c8fdbd8913c4d109da4d725e9e626a6) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:12:42.727812 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc85179-5a13-4087-9595-df6c825af7ab None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:12:42.729494 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc85179-5a13-4087-9595-df6c825af7ab None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:12:42.735009 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 562/1130] 10.222.0.15 () {56 vars in 1374 bytes} [Thu Sep 15 12:12:42 2022] GET /identity/v3/role_assignments?user.id=2c8fdbd8913c4d109da4d725e9e626a6&scope.project.id=b7fb192a30504033bcad23ae818e1e8c&role.id=1081b93ddfd547649ba8582f7a6e719e => generated 608 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:12:44.879551 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6660421-3c67-416e-9f5f-f191af1c251b None None] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:12:44.880078 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6660421-3c67-416e-9f5f-f191af1c251b None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:12:44.880078 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6660421-3c67-416e-9f5f-f191af1c251b None None] PATH_INFO: `/` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:12:44.881527 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 569/1131] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 12:12:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:12:44.886646 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69dcb09c-90a5-4f77-9de4-e5a0854f4f16 None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:12:44.887014 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69dcb09c-90a5-4f77-9de4-e5a0854f4f16 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:12:44.887207 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69dcb09c-90a5-4f77-9de4-e5a0854f4f16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:12:44.906208 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-69dcb09c-90a5-4f77-9de4-e5a0854f4f16 None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:12:44.907414 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-69dcb09c-90a5-4f77-9de4-e5a0854f4f16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:12:44.971829 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 563/1132] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 12:12:44 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 15 12:12:45.238604 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6660421-3c67-416e-9f5f-f191af1c251b None None] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:12:45.287686 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:12:45.336561 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6660421-3c67-416e-9f5f-f191af1c251b None None] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:12:45.368874 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e81582-3e43-49ad-8812-19ed486f11d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:12:45.369498 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e81582-3e43-49ad-8812-19ed486f11d9 None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:12:45.369778 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e81582-3e43-49ad-8812-19ed486f11d9 None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:12:45.370045 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e81582-3e43-49ad-8812-19ed486f11d9 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:12:45.371144 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e81582-3e43-49ad-8812-19ed486f11d9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:12:45.372561 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e81582-3e43-49ad-8812-19ed486f11d9 None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:12:45.374123 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 570/1133] 10.222.0.15 () {56 vars in 1108 bytes} [Thu Sep 15 12:12:45 2022] GET /identity/v3/domains/default => generated 199 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:12:45.380425 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69dcb09c-90a5-4f77-9de4-e5a0854f4f16 None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:12:45.402949 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:12:45.477482 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69dcb09c-90a5-4f77-9de4-e5a0854f4f16 None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:12:45.510745 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef0211f-86db-46f9-96fd-cbdd432eaf81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:12:45.511286 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef0211f-86db-46f9-96fd-cbdd432eaf81 None admin] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:12:45.511562 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef0211f-86db-46f9-96fd-cbdd432eaf81 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:12:45.511832 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef0211f-86db-46f9-96fd-cbdd432eaf81 None admin] PATH_INFO: `/v3/projects` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:12:45.512476 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef0211f-86db-46f9-96fd-cbdd432eaf81 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:12:45.513913 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef0211f-86db-46f9-96fd-cbdd432eaf81 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:12:45.539335 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 564/1134] 10.222.0.15 () {60 vars in 1141 bytes} [Thu Sep 15 12:12:45 2022] POST /identity/v3/projects => generated 311 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 12:12:47.685476 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86eaa2dd-8e75-436d-920a-1dd29e435d6f None None] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:12:47.686046 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86eaa2dd-8e75-436d-920a-1dd29e435d6f None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:12:47.686046 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86eaa2dd-8e75-436d-920a-1dd29e435d6f None None] PATH_INFO: `/` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:12:47.687348 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 571/1135] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 12:12:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:12:47.692464 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc096035-d1f3-4444-a790-de408e4b47a6 None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:12:47.692855 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc096035-d1f3-4444-a790-de408e4b47a6 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:12:47.693041 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc096035-d1f3-4444-a790-de408e4b47a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:12:47.711178 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-dc096035-d1f3-4444-a790-de408e4b47a6 None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:12:47.712349 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-dc096035-d1f3-4444-a790-de408e4b47a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:12:47.776456 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 565/1136] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 12:12:47 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 15 12:12:48.043370 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86eaa2dd-8e75-436d-920a-1dd29e435d6f None None] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:12:48.140309 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86eaa2dd-8e75-436d-920a-1dd29e435d6f None None] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:12:48.156354 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:12:48.174011 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40202ef-7393-4301-ab32-ada41d43cb29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:12:48.174617 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40202ef-7393-4301-ab32-ada41d43cb29 None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:12:48.174892 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40202ef-7393-4301-ab32-ada41d43cb29 None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:12:48.175159 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40202ef-7393-4301-ab32-ada41d43cb29 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:12:48.176239 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40202ef-7393-4301-ab32-ada41d43cb29 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:12:48.177683 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40202ef-7393-4301-ab32-ada41d43cb29 None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:12:48.179194 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 572/1137] 10.222.0.15 () {56 vars in 1108 bytes} [Thu Sep 15 12:12:48 2022] GET /identity/v3/domains/default => generated 199 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:12:48.184766 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc096035-d1f3-4444-a790-de408e4b47a6 None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:12:48.187815 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:12:48.281239 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc096035-d1f3-4444-a790-de408e4b47a6 None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:12:48.314228 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991f74b5-9888-4436-a21d-184008162de8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:12:48.314829 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991f74b5-9888-4436-a21d-184008162de8 None admin] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:12:48.315108 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991f74b5-9888-4436-a21d-184008162de8 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:12:48.315375 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991f74b5-9888-4436-a21d-184008162de8 None admin] PATH_INFO: `/v3/users` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:12:48.316012 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991f74b5-9888-4436-a21d-184008162de8 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:12:48.317458 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991f74b5-9888-4436-a21d-184008162de8 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:12:48.338113 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 566/1138] 10.222.0.15 () {60 vars in 1132 bytes} [Thu Sep 15 12:12:48 2022] POST /identity/v3/users => generated 286 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 12:12:50.506218 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab07d21-912c-403f-a989-f14e997e650f None None] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:12:50.506775 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab07d21-912c-403f-a989-f14e997e650f None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:12:50.506775 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab07d21-912c-403f-a989-f14e997e650f None None] PATH_INFO: `/` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:12:50.508103 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 573/1139] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 12:12:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:12:50.513393 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d420382-548a-4ec8-a228-197d1289952a None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:12:50.513795 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d420382-548a-4ec8-a228-197d1289952a None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:12:50.513932 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d420382-548a-4ec8-a228-197d1289952a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:12:50.531463 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-0d420382-548a-4ec8-a228-197d1289952a None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:12:50.532840 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-0d420382-548a-4ec8-a228-197d1289952a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:12:50.585807 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:12:50.598014 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 567/1140] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 12:12:50 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 15 12:12:50.864310 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab07d21-912c-403f-a989-f14e997e650f None None] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:12:50.964672 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab07d21-912c-403f-a989-f14e997e650f None None] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:12:50.998114 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd54057-086a-43b4-9aac-d232c485b0f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:12:50.998734 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd54057-086a-43b4-9aac-d232c485b0f9 None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:12:50.999010 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd54057-086a-43b4-9aac-d232c485b0f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:12:50.999277 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd54057-086a-43b4-9aac-d232c485b0f9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:12:51.010699 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd54057-086a-43b4-9aac-d232c485b0f9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 12:12:51.015386 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application [None req-9fd54057-086a-43b4-9aac-d232c485b0f9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 15 12:12:51.015386 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:12:51.015386 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:12:51.015386 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:12:51.015386 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:12:51.015386 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:12:51.015386 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:12:51.015386 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:12:51.015386 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:12:51.015386 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:12:51.015386 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:12:51.015386 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:12:51.015386 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 12:12:51.015386 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 12:12:51.015386 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 12:12:51.015386 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 12:12:51.016510 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 12:12:51.016510 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 12:12:51.016510 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 12:12:51.016510 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 12:12:51.016510 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:12:51.016510 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:12:51.016510 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:12:51.016510 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:12:51.016510 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:12:51.016510 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:12:51.016510 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:12:51.016510 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application with Lock( Sep 15 12:12:51.016510 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:12:51.016510 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:12:51.016510 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:12:51.016510 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:12:51.017534 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:12:51.017534 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:12:51.017534 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:12:51.017534 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:12:51.017534 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 12:12:51.017534 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 12:12:51.017534 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 12:12:51.017534 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 12:12:51.017534 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 12:12:51.017534 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 12:12:51.017534 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 15 12:12:51.017534 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application Sep 15 12:12:51.017534 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 574/1141] 10.222.0.15 () {56 vars in 1096 bytes} [Thu Sep 15 12:12:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:12:51.022559 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d420382-548a-4ec8-a228-197d1289952a None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:12:51.117295 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d420382-548a-4ec8-a228-197d1289952a None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:12:51.149350 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab5dac0-bf48-4cef-b44e-a4431805d614 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:12:51.149939 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab5dac0-bf48-4cef-b44e-a4431805d614 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:12:51.150218 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab5dac0-bf48-4cef-b44e-a4431805d614 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:12:51.150486 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab5dac0-bf48-4cef-b44e-a4431805d614 None admin] PATH_INFO: `/v3/roles` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:12:51.151133 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab5dac0-bf48-4cef-b44e-a4431805d614 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:12:51.151431 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab5dac0-bf48-4cef-b44e-a4431805d614 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:12:51.152784 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab5dac0-bf48-4cef-b44e-a4431805d614 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:12:51.161045 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 568/1142] 10.222.0.15 () {56 vars in 1099 bytes} [Thu Sep 15 12:12:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:12:51.166679 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd54057-086a-43b4-9aac-d232c485b0f9 None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:12:51.211703 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:12:51.264906 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd54057-086a-43b4-9aac-d232c485b0f9 None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:12:51.298862 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef328009-49f8-49e5-a96b-61b855319a31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:12:51.299481 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef328009-49f8-49e5-a96b-61b855319a31 None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:12:51.299760 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef328009-49f8-49e5-a96b-61b855319a31 None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:12:51.300033 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef328009-49f8-49e5-a96b-61b855319a31 None admin] PATH_INFO: `/v3/users/e74ac62c4a624135ae7d2565db545e01` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:12:51.312552 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef328009-49f8-49e5-a96b-61b855319a31 None admin] RBAC: Authorizing `identity:get_user(user_id=e74ac62c4a624135ae7d2565db545e01)` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:12:51.314126 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef328009-49f8-49e5-a96b-61b855319a31 None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:12:51.315684 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 575/1143] 10.222.0.15 () {56 vars in 1177 bytes} [Thu Sep 15 12:12:51 2022] GET /identity/v3/users/e74ac62c4a624135ae7d2565db545e01 => generated 286 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:12:51.321388 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab5dac0-bf48-4cef-b44e-a4431805d614 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:12:51.373622 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:12:51.419994 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab5dac0-bf48-4cef-b44e-a4431805d614 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:12:51.451009 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a8b740-f37e-4527-970c-8856a2c22429 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:12:51.451603 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a8b740-f37e-4527-970c-8856a2c22429 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:12:51.451879 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a8b740-f37e-4527-970c-8856a2c22429 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:12:51.452146 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a8b740-f37e-4527-970c-8856a2c22429 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:12:51.518391 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a8b740-f37e-4527-970c-8856a2c22429 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:12:51.519891 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a8b740-f37e-4527-970c-8856a2c22429 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:12:51.567663 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:12:51.613091 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 569/1144] 10.222.0.15 () {58 vars in 1282 bytes} [Thu Sep 15 12:12:51 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 15 12:12:51.618955 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef328009-49f8-49e5-a96b-61b855319a31 None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:12:51.714585 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef328009-49f8-49e5-a96b-61b855319a31 None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:12:51.716052 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:12:51.749341 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f421d851-7f4e-4894-b16b-dda47991a087 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:12:51.749961 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f421d851-7f4e-4894-b16b-dda47991a087 None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:12:51.750239 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f421d851-7f4e-4894-b16b-dda47991a087 None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:12:51.750532 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f421d851-7f4e-4894-b16b-dda47991a087 None admin] PATH_INFO: `/v3/projects/fa47700a39c744a9ae5e447e202f95d9` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:12:51.751637 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f421d851-7f4e-4894-b16b-dda47991a087 None admin] RBAC: Authorizing `identity:get_project(project_id=fa47700a39c744a9ae5e447e202f95d9)` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:12:51.753084 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f421d851-7f4e-4894-b16b-dda47991a087 None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:12:51.754664 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 576/1145] 10.222.0.15 () {56 vars in 1186 bytes} [Thu Sep 15 12:12:51 2022] GET /identity/v3/projects/fa47700a39c744a9ae5e447e202f95d9 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:12:51.760916 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a8b740-f37e-4527-970c-8856a2c22429 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:12:51.854002 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a8b740-f37e-4527-970c-8856a2c22429 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:12:51.885823 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601267c3-cf54-42b2-810b-feffb3af925e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:12:51.886410 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601267c3-cf54-42b2-810b-feffb3af925e None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:12:51.886687 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601267c3-cf54-42b2-810b-feffb3af925e None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:12:51.886953 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601267c3-cf54-42b2-810b-feffb3af925e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:12:51.887651 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601267c3-cf54-42b2-810b-feffb3af925e None admin] RBAC: Adding query filter params (role.id=9c769b5c82884766b2c510b1f0f1364c, scope.project.id=fa47700a39c744a9ae5e447e202f95d9, user.id=e74ac62c4a624135ae7d2565db545e01) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:12:51.887955 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601267c3-cf54-42b2-810b-feffb3af925e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:12:51.889371 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601267c3-cf54-42b2-810b-feffb3af925e None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:12:51.895144 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 570/1146] 10.222.0.15 () {56 vars in 1374 bytes} [Thu Sep 15 12:12:51 2022] GET /identity/v3/role_assignments?user.id=e74ac62c4a624135ae7d2565db545e01&scope.project.id=fa47700a39c744a9ae5e447e202f95d9&role.id=9c769b5c82884766b2c510b1f0f1364c => generated 259 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:12:53.999676 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a41e970-5672-418c-93d1-9f110e3e0ac7 None None] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:12:54.000137 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a41e970-5672-418c-93d1-9f110e3e0ac7 None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:12:54.000230 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a41e970-5672-418c-93d1-9f110e3e0ac7 None None] PATH_INFO: `/` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:12:54.002076 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 577/1147] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 12:12:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:12:54.007265 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504bee66-277f-4676-8f74-f733a90fb644 None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:12:54.007689 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504bee66-277f-4676-8f74-f733a90fb644 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:12:54.007794 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504bee66-277f-4676-8f74-f733a90fb644 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:12:54.026012 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-504bee66-277f-4676-8f74-f733a90fb644 None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:12:54.027173 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-504bee66-277f-4676-8f74-f733a90fb644 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:12:54.037336 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:12:54.091969 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 571/1148] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 12:12:54 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 15 12:12:54.356060 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a41e970-5672-418c-93d1-9f110e3e0ac7 None None] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:12:54.453021 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a41e970-5672-418c-93d1-9f110e3e0ac7 None None] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:12:54.484731 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c28bcf8-b6b1-4f45-b725-b47016bceddb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:12:54.485908 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c28bcf8-b6b1-4f45-b725-b47016bceddb None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:12:54.486184 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c28bcf8-b6b1-4f45-b725-b47016bceddb None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:12:54.486451 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c28bcf8-b6b1-4f45-b725-b47016bceddb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:12:54.496053 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c28bcf8-b6b1-4f45-b725-b47016bceddb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 12:12:54.501327 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application [None req-1c28bcf8-b6b1-4f45-b725-b47016bceddb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 15 12:12:54.501327 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:12:54.501327 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:12:54.501327 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:12:54.501327 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:12:54.501327 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:12:54.501327 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:12:54.501327 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:12:54.501327 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:12:54.501327 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:12:54.501327 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:12:54.501327 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:12:54.501327 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 12:12:54.501327 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 12:12:54.501327 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 12:12:54.501327 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 12:12:54.502535 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 12:12:54.502535 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 12:12:54.502535 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 12:12:54.502535 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 12:12:54.502535 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:12:54.502535 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:12:54.502535 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:12:54.502535 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:12:54.502535 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:12:54.502535 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:12:54.502535 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:12:54.502535 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application with Lock( Sep 15 12:12:54.502535 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:12:54.502535 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:12:54.502535 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:12:54.502535 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:12:54.503544 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:12:54.503544 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:12:54.503544 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:12:54.503544 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:12:54.503544 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 12:12:54.503544 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 12:12:54.503544 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 12:12:54.503544 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 12:12:54.503544 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 12:12:54.503544 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 12:12:54.503544 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 15 12:12:54.503544 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application Sep 15 12:12:54.503544 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 578/1149] 10.222.0.15 () {56 vars in 1096 bytes} [Thu Sep 15 12:12:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 147 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:12:54.508184 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504bee66-277f-4676-8f74-f733a90fb644 None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:12:54.601605 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504bee66-277f-4676-8f74-f733a90fb644 None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:12:54.634158 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59bd422-27ff-43b7-b753-1b94f735df26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:12:54.634698 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59bd422-27ff-43b7-b753-1b94f735df26 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:12:54.634973 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59bd422-27ff-43b7-b753-1b94f735df26 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:12:54.635242 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59bd422-27ff-43b7-b753-1b94f735df26 None admin] PATH_INFO: `/v3/roles` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:12:54.635884 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e59bd422-27ff-43b7-b753-1b94f735df26 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:12:54.636181 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e59bd422-27ff-43b7-b753-1b94f735df26 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:12:54.637554 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e59bd422-27ff-43b7-b753-1b94f735df26 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:12:54.645174 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 572/1150] 10.222.0.15 () {56 vars in 1099 bytes} [Thu Sep 15 12:12:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:12:54.650658 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c28bcf8-b6b1-4f45-b725-b47016bceddb None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:12:54.680832 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:12:54.746838 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c28bcf8-b6b1-4f45-b725-b47016bceddb None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:12:54.778517 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c024b9-7d10-4145-b50e-5a86132f7a20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:12:54.779131 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c024b9-7d10-4145-b50e-5a86132f7a20 None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:12:54.779404 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c024b9-7d10-4145-b50e-5a86132f7a20 None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:12:54.779675 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c024b9-7d10-4145-b50e-5a86132f7a20 None admin] PATH_INFO: `/v3/users/e74ac62c4a624135ae7d2565db545e01` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:12:54.780786 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c024b9-7d10-4145-b50e-5a86132f7a20 None admin] RBAC: Authorizing `identity:get_user(user_id=e74ac62c4a624135ae7d2565db545e01)` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:12:54.782732 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c024b9-7d10-4145-b50e-5a86132f7a20 None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:12:54.784252 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 579/1151] 10.222.0.15 () {56 vars in 1177 bytes} [Thu Sep 15 12:12:54 2022] GET /identity/v3/users/e74ac62c4a624135ae7d2565db545e01 => generated 286 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:12:54.789978 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59bd422-27ff-43b7-b753-1b94f735df26 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:12:54.801590 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:12:54.886295 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59bd422-27ff-43b7-b753-1b94f735df26 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:12:54.919071 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f37cacd-0eba-4167-a9e0-7a9589969c36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:12:54.919681 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f37cacd-0eba-4167-a9e0-7a9589969c36 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:12:54.919955 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f37cacd-0eba-4167-a9e0-7a9589969c36 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:12:54.920220 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f37cacd-0eba-4167-a9e0-7a9589969c36 None admin] PATH_INFO: `/v3/projects/fa47700a39c744a9ae5e447e202f95d9` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:12:54.921319 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f37cacd-0eba-4167-a9e0-7a9589969c36 None admin] RBAC: Authorizing `identity:get_project(project_id=fa47700a39c744a9ae5e447e202f95d9)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:12:54.922769 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f37cacd-0eba-4167-a9e0-7a9589969c36 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:12:54.924328 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 573/1152] 10.222.0.15 () {56 vars in 1186 bytes} [Thu Sep 15 12:12:54 2022] GET /identity/v3/projects/fa47700a39c744a9ae5e447e202f95d9 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:12:54.930262 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c024b9-7d10-4145-b50e-5a86132f7a20 None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:12:55.023223 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:12:55.027250 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c024b9-7d10-4145-b50e-5a86132f7a20 None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:12:55.060179 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3337ccf8-2f0b-416b-bba1-ad87caa1bbc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:12:55.060852 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3337ccf8-2f0b-416b-bba1-ad87caa1bbc0 None admin] REQUEST_METHOD: `PUT` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:12:55.061131 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3337ccf8-2f0b-416b-bba1-ad87caa1bbc0 None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:12:55.061402 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3337ccf8-2f0b-416b-bba1-ad87caa1bbc0 None admin] PATH_INFO: `/v3/projects/fa47700a39c744a9ae5e447e202f95d9/users/e74ac62c4a624135ae7d2565db545e01/roles/9c769b5c82884766b2c510b1f0f1364c` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:12:55.063290 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3337ccf8-2f0b-416b-bba1-ad87caa1bbc0 None admin] RBAC: Authorizing `identity:create_grant(project_id=fa47700a39c744a9ae5e447e202f95d9, user_id=e74ac62c4a624135ae7d2565db545e01, role_id=9c769b5c82884766b2c510b1f0f1364c)` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:12:55.064775 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3337ccf8-2f0b-416b-bba1-ad87caa1bbc0 None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:12:55.076994 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 580/1153] 10.222.0.15 () {58 vars in 1439 bytes} [Thu Sep 15 12:12:54 2022] PUT /identity/v3/projects/fa47700a39c744a9ae5e447e202f95d9/users/e74ac62c4a624135ae7d2565db545e01/roles/9c769b5c82884766b2c510b1f0f1364c => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:12:57.165848 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60cde68-bcec-4ca1-ba92-cafe7ba7a4cf None None] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:12:57.166397 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60cde68-bcec-4ca1-ba92-cafe7ba7a4cf None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:12:57.166397 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60cde68-bcec-4ca1-ba92-cafe7ba7a4cf None None] PATH_INFO: `/` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:12:57.167709 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 574/1154] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 12:12:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:12:57.173335 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d6ffbc-c01f-4c73-a6fa-45ca5779e788 None None] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:12:57.173752 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d6ffbc-c01f-4c73-a6fa-45ca5779e788 None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:12:57.173867 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d6ffbc-c01f-4c73-a6fa-45ca5779e788 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:12:57.191957 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.auth.core [None req-19d6ffbc-c01f-4c73-a6fa-45ca5779e788 None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81449) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:12:57.193148 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-19d6ffbc-c01f-4c73-a6fa-45ca5779e788 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:12:57.261232 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 581/1155] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 12:12:57 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 15 12:12:57.532327 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60cde68-bcec-4ca1-ba92-cafe7ba7a4cf None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:12:57.608933 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:12:57.632807 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60cde68-bcec-4ca1-ba92-cafe7ba7a4cf None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:12:57.664636 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da012d65-b533-4559-ac43-c28a0545f9c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:12:57.665283 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da012d65-b533-4559-ac43-c28a0545f9c7 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:12:57.665561 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da012d65-b533-4559-ac43-c28a0545f9c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:12:57.665829 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da012d65-b533-4559-ac43-c28a0545f9c7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:12:57.676002 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da012d65-b533-4559-ac43-c28a0545f9c7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 12:12:57.680840 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application [None req-da012d65-b533-4559-ac43-c28a0545f9c7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 15 12:12:57.680840 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:12:57.680840 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:12:57.680840 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:12:57.680840 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:12:57.680840 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:12:57.680840 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:12:57.680840 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:12:57.680840 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:12:57.680840 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:12:57.680840 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:12:57.680840 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:12:57.680840 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 12:12:57.680840 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 12:12:57.680840 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 12:12:57.680840 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 12:12:57.681910 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 12:12:57.681910 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 12:12:57.681910 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 12:12:57.681910 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 12:12:57.681910 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:12:57.681910 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:12:57.681910 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:12:57.681910 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:12:57.681910 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:12:57.681910 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:12:57.681910 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:12:57.681910 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application with Lock( Sep 15 12:12:57.681910 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:12:57.681910 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:12:57.681910 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:12:57.681910 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:12:57.683044 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:12:57.683044 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:12:57.683044 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:12:57.683044 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:12:57.683044 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 12:12:57.683044 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 12:12:57.683044 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 12:12:57.683044 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 12:12:57.683044 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 12:12:57.683044 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 12:12:57.683044 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 15 12:12:57.683044 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 15 12:12:57.683044 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 575/1156] 10.222.0.15 () {56 vars in 1096 bytes} [Thu Sep 15 12:12:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:12:57.687936 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d6ffbc-c01f-4c73-a6fa-45ca5779e788 None None] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:12:57.770751 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:12:57.783752 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d6ffbc-c01f-4c73-a6fa-45ca5779e788 None None] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:12:57.815993 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1527c2-86ce-4981-982b-a015dfcf1982 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:12:57.816591 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1527c2-86ce-4981-982b-a015dfcf1982 None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:12:57.816891 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1527c2-86ce-4981-982b-a015dfcf1982 None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:12:57.817157 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1527c2-86ce-4981-982b-a015dfcf1982 None admin] PATH_INFO: `/v3/roles` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:12:57.817802 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f1527c2-86ce-4981-982b-a015dfcf1982 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81449) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:12:57.818097 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f1527c2-86ce-4981-982b-a015dfcf1982 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:12:57.819433 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f1527c2-86ce-4981-982b-a015dfcf1982 None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:12:57.827043 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 582/1157] 10.222.0.15 () {56 vars in 1099 bytes} [Thu Sep 15 12:12:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:12:57.832910 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da012d65-b533-4559-ac43-c28a0545f9c7 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:12:57.928421 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da012d65-b533-4559-ac43-c28a0545f9c7 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:12:57.960579 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a39452-a241-40e2-8ba1-6792c5cd627a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:12:57.961229 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a39452-a241-40e2-8ba1-6792c5cd627a None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:12:57.961507 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a39452-a241-40e2-8ba1-6792c5cd627a None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:12:57.961776 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a39452-a241-40e2-8ba1-6792c5cd627a None admin] PATH_INFO: `/v3/users/e74ac62c4a624135ae7d2565db545e01` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:12:57.962888 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a39452-a241-40e2-8ba1-6792c5cd627a None admin] RBAC: Authorizing `identity:get_user(user_id=e74ac62c4a624135ae7d2565db545e01)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:12:57.964294 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a39452-a241-40e2-8ba1-6792c5cd627a None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:12:57.965884 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 576/1158] 10.222.0.15 () {56 vars in 1177 bytes} [Thu Sep 15 12:12:57 2022] GET /identity/v3/users/e74ac62c4a624135ae7d2565db545e01 => generated 286 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:12:57.972131 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1527c2-86ce-4981-982b-a015dfcf1982 None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:12:58.066921 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1527c2-86ce-4981-982b-a015dfcf1982 None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:12:58.098996 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336e480e-21af-4bc9-aae1-16e5d41dfd5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:12:58.099580 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336e480e-21af-4bc9-aae1-16e5d41dfd5c None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:12:58.099853 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336e480e-21af-4bc9-aae1-16e5d41dfd5c None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:12:58.100119 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336e480e-21af-4bc9-aae1-16e5d41dfd5c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:12:58.117767 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:12:58.164432 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336e480e-21af-4bc9-aae1-16e5d41dfd5c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:12:58.165908 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336e480e-21af-4bc9-aae1-16e5d41dfd5c None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:12:58.257930 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 583/1159] 10.222.0.15 () {58 vars in 1282 bytes} [Thu Sep 15 12:12:57 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 286 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 15 12:12:58.263784 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a39452-a241-40e2-8ba1-6792c5cd627a None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:12:58.268204 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:12:58.358322 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a39452-a241-40e2-8ba1-6792c5cd627a None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:12:58.390263 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6f09e5-95e3-42ac-a7a5-4b51a862982a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:12:58.391417 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6f09e5-95e3-42ac-a7a5-4b51a862982a None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:12:58.391696 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6f09e5-95e3-42ac-a7a5-4b51a862982a None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:12:58.391960 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6f09e5-95e3-42ac-a7a5-4b51a862982a None admin] PATH_INFO: `/v3/projects/fa47700a39c744a9ae5e447e202f95d9` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:12:58.393064 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6f09e5-95e3-42ac-a7a5-4b51a862982a None admin] RBAC: Authorizing `identity:get_project(project_id=fa47700a39c744a9ae5e447e202f95d9)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:12:58.394465 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6f09e5-95e3-42ac-a7a5-4b51a862982a None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:12:58.396034 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 577/1160] 10.222.0.15 () {56 vars in 1186 bytes} [Thu Sep 15 12:12:58 2022] GET /identity/v3/projects/fa47700a39c744a9ae5e447e202f95d9 => generated 311 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:12:58.402027 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336e480e-21af-4bc9-aae1-16e5d41dfd5c None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:12:58.448681 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:12:58.495912 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336e480e-21af-4bc9-aae1-16e5d41dfd5c None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:12:58.528549 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add8b58d-b9cf-40c2-a2c2-350b1e9eb15b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:12:58.529156 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add8b58d-b9cf-40c2-a2c2-350b1e9eb15b None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:12:58.529432 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add8b58d-b9cf-40c2-a2c2-350b1e9eb15b None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:12:58.529696 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add8b58d-b9cf-40c2-a2c2-350b1e9eb15b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:12:58.530392 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add8b58d-b9cf-40c2-a2c2-350b1e9eb15b None admin] RBAC: Adding query filter params (role.id=9c769b5c82884766b2c510b1f0f1364c, scope.project.id=fa47700a39c744a9ae5e447e202f95d9, user.id=e74ac62c4a624135ae7d2565db545e01) {{(pid=81449) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:12:58.530690 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add8b58d-b9cf-40c2-a2c2-350b1e9eb15b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:12:58.532110 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add8b58d-b9cf-40c2-a2c2-350b1e9eb15b None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:12:58.537536 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 584/1161] 10.222.0.15 () {56 vars in 1374 bytes} [Thu Sep 15 12:12:58 2022] GET /identity/v3/role_assignments?user.id=e74ac62c4a624135ae7d2565db545e01&scope.project.id=fa47700a39c744a9ae5e447e202f95d9&role.id=9c769b5c82884766b2c510b1f0f1364c => generated 608 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:13:00.705077 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba70fba-b859-4535-ad48-1dd754961376 None None] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:13:00.705674 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba70fba-b859-4535-ad48-1dd754961376 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:13:00.705674 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba70fba-b859-4535-ad48-1dd754961376 None None] PATH_INFO: `/` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:13:00.706945 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 578/1162] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 12:13:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:13:00.712275 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2cbe53-d5df-4c35-8983-e4d77f7c356f None None] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:13:00.712692 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2cbe53-d5df-4c35-8983-e4d77f7c356f None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:13:00.712852 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2cbe53-d5df-4c35-8983-e4d77f7c356f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:13:00.731349 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.auth.core [None req-db2cbe53-d5df-4c35-8983-e4d77f7c356f None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81449) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:13:00.732525 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-db2cbe53-d5df-4c35-8983-e4d77f7c356f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:13:00.798171 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 585/1163] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 12:13:00 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 15 12:13:01.065667 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba70fba-b859-4535-ad48-1dd754961376 None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:13:01.134369 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:13:01.164942 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba70fba-b859-4535-ad48-1dd754961376 None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:13:01.199939 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f373169-dc6a-4384-a462-6df01e8f267e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:13:01.200545 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f373169-dc6a-4384-a462-6df01e8f267e None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:13:01.200851 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f373169-dc6a-4384-a462-6df01e8f267e None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:13:01.201126 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f373169-dc6a-4384-a462-6df01e8f267e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:13:01.267505 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f373169-dc6a-4384-a462-6df01e8f267e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:13:01.268908 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f373169-dc6a-4384-a462-6df01e8f267e None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:13:01.333707 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:13:01.362132 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 579/1164] 10.222.0.15 () {58 vars in 1282 bytes} [Thu Sep 15 12:13:01 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 15 12:13:01.368023 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2cbe53-d5df-4c35-8983-e4d77f7c356f None None] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:13:01.412396 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:13:01.464863 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2cbe53-d5df-4c35-8983-e4d77f7c356f None None] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:13:01.498248 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c743be0a-0b35-4236-9f05-7443c9f5d526 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:13:01.498873 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c743be0a-0b35-4236-9f05-7443c9f5d526 None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:13:01.499149 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c743be0a-0b35-4236-9f05-7443c9f5d526 None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:13:01.499417 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c743be0a-0b35-4236-9f05-7443c9f5d526 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:13:01.506395 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c743be0a-0b35-4236-9f05-7443c9f5d526 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:13:01.507819 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c743be0a-0b35-4236-9f05-7443c9f5d526 None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:13:01.519481 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application [None req-c743be0a-0b35-4236-9f05-7443c9f5d526 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Sep 15 12:13:01.519481 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:13:01.519481 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Sep 15 12:13:01.519481 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Sep 15 12:13:01.519481 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 15 12:13:01.519481 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 15 12:13:01.519481 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 15 12:13:01.519481 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 15 12:13:01.519481 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Sep 15 12:13:01.519481 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application Sep 15 12:13:01.519481 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 15 12:13:01.519481 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application Sep 15 12:13:01.519481 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:13:01.519481 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:13:01.519481 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:13:01.519481 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:13:01.519481 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:13:01.520676 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:13:01.520676 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:13:01.520676 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:13:01.520676 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:13:01.520676 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:13:01.520676 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:13:01.520676 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 12:13:01.520676 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 12:13:01.520676 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 12:13:01.520676 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 12:13:01.520676 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:13:01.520676 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:13:01.520676 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:13:01.520676 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:13:01.520676 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:13:01.520676 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:13:01.521709 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:13:01.521709 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application with Lock( Sep 15 12:13:01.521709 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:13:01.521709 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:13:01.521709 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:13:01.521709 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:13:01.521709 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:13:01.521709 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:13:01.521709 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:13:01.521709 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:13:01.521709 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Sep 15 12:13:01.521709 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 12:13:01.521709 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Sep 15 12:13:01.521709 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application Sep 15 12:13:01.521709 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 586/1165] 10.222.0.15 () {56 vars in 1117 bytes} [Thu Sep 15 12:13:01 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:13:01.526520 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f373169-dc6a-4384-a462-6df01e8f267e None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:13:01.620800 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f373169-dc6a-4384-a462-6df01e8f267e None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:13:01.653583 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd0c858-18e5-467e-a014-b1d75aae2047 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:13:01.654178 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd0c858-18e5-467e-a014-b1d75aae2047 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:13:01.654457 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd0c858-18e5-467e-a014-b1d75aae2047 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:13:01.654726 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd0c858-18e5-467e-a014-b1d75aae2047 None admin] PATH_INFO: `/v3/domains` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:13:01.655354 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd0c858-18e5-467e-a014-b1d75aae2047 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:13:01.655653 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd0c858-18e5-467e-a014-b1d75aae2047 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:13:01.657051 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd0c858-18e5-467e-a014-b1d75aae2047 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:13:01.663847 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 580/1166] 10.222.0.15 () {56 vars in 1115 bytes} [Thu Sep 15 12:13:01 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:13:03.783688 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa97596-06a5-4d01-ace1-4da469c9b27b None None] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:13:03.784246 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa97596-06a5-4d01-ace1-4da469c9b27b None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:13:03.784246 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa97596-06a5-4d01-ace1-4da469c9b27b None None] PATH_INFO: `/` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:13:03.785619 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 587/1167] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 12:13:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:13:03.790857 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330e5b77-4662-4c54-a147-72d7f97fec2d None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:13:03.791244 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330e5b77-4662-4c54-a147-72d7f97fec2d None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:13:03.791477 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330e5b77-4662-4c54-a147-72d7f97fec2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:13:03.809255 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-330e5b77-4662-4c54-a147-72d7f97fec2d None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:13:03.810432 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-330e5b77-4662-4c54-a147-72d7f97fec2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:13:03.838972 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:13:03.875904 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 581/1168] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 12:13:03 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 15 12:13:04.145656 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa97596-06a5-4d01-ace1-4da469c9b27b None None] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:13:04.243371 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa97596-06a5-4d01-ace1-4da469c9b27b None None] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:13:04.245647 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:13:04.276442 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b33d17-76d9-4289-a359-c247317e2c2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:13:04.277066 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b33d17-76d9-4289-a359-c247317e2c2b None admin] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:13:04.277344 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b33d17-76d9-4289-a359-c247317e2c2b None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:13:04.277612 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b33d17-76d9-4289-a359-c247317e2c2b None admin] PATH_INFO: `/v3/domains` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:13:04.278263 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b33d17-76d9-4289-a359-c247317e2c2b None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:13:04.279698 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b33d17-76d9-4289-a359-c247317e2c2b None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:13:04.294812 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 588/1169] 10.222.0.15 () {60 vars in 1138 bytes} [Thu Sep 15 12:13:04 2022] POST /identity/v3/domains => generated 267 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 12:13:06.441972 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddffd280-a0e6-4ce2-8b04-020728ed28b6 None None] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:13:06.442481 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddffd280-a0e6-4ce2-8b04-020728ed28b6 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:13:06.442584 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddffd280-a0e6-4ce2-8b04-020728ed28b6 None None] PATH_INFO: `/` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:13:06.443860 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 582/1170] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 12:13:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:13:06.449000 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea853c1-0563-4326-8617-300d42ffb632 None None] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:13:06.449428 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea853c1-0563-4326-8617-300d42ffb632 None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:13:06.449536 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea853c1-0563-4326-8617-300d42ffb632 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:13:06.467550 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.auth.core [None req-cea853c1-0563-4326-8617-300d42ffb632 None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81449) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:13:06.469381 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-cea853c1-0563-4326-8617-300d42ffb632 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:13:06.533349 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 589/1171] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 12:13:06 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 15 12:13:06.804029 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddffd280-a0e6-4ce2-8b04-020728ed28b6 None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:13:06.904335 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddffd280-a0e6-4ce2-8b04-020728ed28b6 None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:13:06.937917 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18913d44-a785-4bf8-9149-e8a186982529 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:13:06.938545 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18913d44-a785-4bf8-9149-e8a186982529 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:13:06.938827 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18913d44-a785-4bf8-9149-e8a186982529 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:13:06.939097 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18913d44-a785-4bf8-9149-e8a186982529 None admin] PATH_INFO: `/v3/domains/b12b9237ac084fad8ca22ac6741188a1` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:13:06.949138 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18913d44-a785-4bf8-9149-e8a186982529 None admin] RBAC: Authorizing `identity:get_domain(domain_id=b12b9237ac084fad8ca22ac6741188a1)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:13:06.950584 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18913d44-a785-4bf8-9149-e8a186982529 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:13:06.952124 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 583/1172] 10.222.0.15 () {56 vars in 1183 bytes} [Thu Sep 15 12:13:06 2022] GET /identity/v3/domains/b12b9237ac084fad8ca22ac6741188a1 => generated 267 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:13:06.958009 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea853c1-0563-4326-8617-300d42ffb632 None None] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:13:07.047057 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:13:07.054846 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea853c1-0563-4326-8617-300d42ffb632 None None] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:13:07.087404 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ec2441-346c-4a4c-aae4-74d77d041042 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:13:07.088011 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ec2441-346c-4a4c-aae4-74d77d041042 None admin] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:13:07.088290 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ec2441-346c-4a4c-aae4-74d77d041042 None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:13:07.088561 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ec2441-346c-4a4c-aae4-74d77d041042 None admin] PATH_INFO: `/v3/projects` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:13:07.089226 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ec2441-346c-4a4c-aae4-74d77d041042 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:13:07.090645 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ec2441-346c-4a4c-aae4-74d77d041042 None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:13:07.116078 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 590/1173] 10.222.0.15 () {60 vars in 1141 bytes} [Thu Sep 15 12:13:06 2022] POST /identity/v3/projects => generated 361 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 12:13:09.353951 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb2f1c5-f172-4380-8383-c5fdcd122e89 None None] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:13:09.354446 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb2f1c5-f172-4380-8383-c5fdcd122e89 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:13:09.354446 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb2f1c5-f172-4380-8383-c5fdcd122e89 None None] PATH_INFO: `/` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:13:09.355730 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 584/1174] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 12:13:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:13:09.361689 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef19ff4-c35e-4d0b-837c-e6d2f161f45f None None] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:13:09.362111 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef19ff4-c35e-4d0b-837c-e6d2f161f45f None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:13:09.362233 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef19ff4-c35e-4d0b-837c-e6d2f161f45f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:13:09.380491 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.auth.core [None req-3ef19ff4-c35e-4d0b-837c-e6d2f161f45f None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81449) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:13:09.381703 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-3ef19ff4-c35e-4d0b-837c-e6d2f161f45f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:13:09.446802 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 591/1175] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 12:13:09 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 15 12:13:09.713761 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb2f1c5-f172-4380-8383-c5fdcd122e89 None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:13:09.742017 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:13:09.813082 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb2f1c5-f172-4380-8383-c5fdcd122e89 None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:13:09.845123 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569b3974-ef70-4678-b9a7-2e8a84da42a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:13:09.845747 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569b3974-ef70-4678-b9a7-2e8a84da42a6 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:13:09.846025 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569b3974-ef70-4678-b9a7-2e8a84da42a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:13:09.847402 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569b3974-ef70-4678-b9a7-2e8a84da42a6 None admin] PATH_INFO: `/v3/domains/b12b9237ac084fad8ca22ac6741188a1` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:13:09.847402 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569b3974-ef70-4678-b9a7-2e8a84da42a6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=b12b9237ac084fad8ca22ac6741188a1)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:13:09.848816 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569b3974-ef70-4678-b9a7-2e8a84da42a6 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:13:09.850365 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 585/1176] 10.222.0.15 () {56 vars in 1183 bytes} [Thu Sep 15 12:13:09 2022] GET /identity/v3/domains/b12b9237ac084fad8ca22ac6741188a1 => generated 267 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:13:09.856501 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef19ff4-c35e-4d0b-837c-e6d2f161f45f None None] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:13:09.930751 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:13:09.954147 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef19ff4-c35e-4d0b-837c-e6d2f161f45f None None] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:13:09.987470 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96818039-4049-4ec1-9ce6-da33310e382b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:13:09.988074 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96818039-4049-4ec1-9ce6-da33310e382b None admin] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:13:09.988351 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96818039-4049-4ec1-9ce6-da33310e382b None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:13:09.988617 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96818039-4049-4ec1-9ce6-da33310e382b None admin] PATH_INFO: `/v3/users` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:13:09.989296 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96818039-4049-4ec1-9ce6-da33310e382b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:13:09.990718 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96818039-4049-4ec1-9ce6-da33310e382b None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:13:10.010760 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 592/1177] 10.222.0.15 () {60 vars in 1132 bytes} [Thu Sep 15 12:13:09 2022] POST /identity/v3/users => generated 311 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 12:13:12.155802 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e34ca7-9c0d-4f72-aadb-11a0fca2a223 None None] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:13:12.156353 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e34ca7-9c0d-4f72-aadb-11a0fca2a223 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:13:12.156353 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e34ca7-9c0d-4f72-aadb-11a0fca2a223 None None] PATH_INFO: `/` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:13:12.157831 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 586/1178] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 12:13:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:13:12.162956 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274d76cd-992d-4889-9c2d-b027fa1a7f75 None None] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:13:12.163341 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274d76cd-992d-4889-9c2d-b027fa1a7f75 None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:13:12.163495 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274d76cd-992d-4889-9c2d-b027fa1a7f75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:13:12.181564 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.auth.core [None req-274d76cd-992d-4889-9c2d-b027fa1a7f75 None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81449) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:13:12.182717 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-274d76cd-992d-4889-9c2d-b027fa1a7f75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:13:12.245982 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 593/1179] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 12:13:12 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 15 12:13:12.514112 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e34ca7-9c0d-4f72-aadb-11a0fca2a223 None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:13:12.604934 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:13:12.613101 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e34ca7-9c0d-4f72-aadb-11a0fca2a223 None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:13:12.645062 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7094f9b8-18ae-4f88-8245-00f3476d4121 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:13:12.645681 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7094f9b8-18ae-4f88-8245-00f3476d4121 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:13:12.645955 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7094f9b8-18ae-4f88-8245-00f3476d4121 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:13:12.646243 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7094f9b8-18ae-4f88-8245-00f3476d4121 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:13:12.656279 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7094f9b8-18ae-4f88-8245-00f3476d4121 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 12:13:12.661017 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application [None req-7094f9b8-18ae-4f88-8245-00f3476d4121 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 15 12:13:12.661017 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:13:12.661017 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:13:12.661017 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:13:12.661017 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:13:12.661017 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:13:12.661017 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:13:12.661017 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:13:12.661017 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:13:12.661017 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:13:12.661017 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:13:12.661017 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:13:12.661017 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 12:13:12.661017 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 12:13:12.661017 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 12:13:12.661017 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 12:13:12.662090 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 12:13:12.662090 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 12:13:12.662090 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 12:13:12.662090 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 12:13:12.662090 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:13:12.662090 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:13:12.662090 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:13:12.662090 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:13:12.662090 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:13:12.662090 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:13:12.662090 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:13:12.662090 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application with Lock( Sep 15 12:13:12.662090 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:13:12.662090 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:13:12.662090 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:13:12.662090 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:13:12.663282 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:13:12.663282 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:13:12.663282 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:13:12.663282 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:13:12.663282 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 12:13:12.663282 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 12:13:12.663282 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 12:13:12.663282 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 12:13:12.663282 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 12:13:12.663282 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 12:13:12.663282 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 15 12:13:12.663282 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 15 12:13:12.663282 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 587/1180] 10.222.0.15 () {56 vars in 1096 bytes} [Thu Sep 15 12:13:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:13:12.667999 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274d76cd-992d-4889-9c2d-b027fa1a7f75 None None] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:13:12.725516 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:13:12.763392 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274d76cd-992d-4889-9c2d-b027fa1a7f75 None None] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:13:12.795587 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc0b642-a45f-47e5-99b5-624bed9f457c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:13:12.796120 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc0b642-a45f-47e5-99b5-624bed9f457c None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:13:12.796965 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc0b642-a45f-47e5-99b5-624bed9f457c None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:13:12.797232 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc0b642-a45f-47e5-99b5-624bed9f457c None admin] PATH_INFO: `/v3/roles` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:13:12.797878 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc0b642-a45f-47e5-99b5-624bed9f457c None admin] RBAC: Adding query filter params (name=admin) {{(pid=81449) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:13:12.798173 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc0b642-a45f-47e5-99b5-624bed9f457c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:13:12.799512 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc0b642-a45f-47e5-99b5-624bed9f457c None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:13:12.807129 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 594/1181] 10.222.0.15 () {56 vars in 1099 bytes} [Thu Sep 15 12:13:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:13:12.813006 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7094f9b8-18ae-4f88-8245-00f3476d4121 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:13:12.909316 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7094f9b8-18ae-4f88-8245-00f3476d4121 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:13:12.941929 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b29222e-8010-4222-815b-d370f9405f96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:13:12.942555 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b29222e-8010-4222-815b-d370f9405f96 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:13:12.942833 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b29222e-8010-4222-815b-d370f9405f96 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:13:12.943105 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b29222e-8010-4222-815b-d370f9405f96 None admin] PATH_INFO: `/v3/users/e50b1637d55b4d45bada99bf95cfdff7` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:13:12.956094 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b29222e-8010-4222-815b-d370f9405f96 None admin] RBAC: Authorizing `identity:get_user(user_id=e50b1637d55b4d45bada99bf95cfdff7)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:13:12.957577 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b29222e-8010-4222-815b-d370f9405f96 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:13:12.959171 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 588/1182] 10.222.0.15 () {56 vars in 1177 bytes} [Thu Sep 15 12:13:12 2022] GET /identity/v3/users/e50b1637d55b4d45bada99bf95cfdff7 => generated 311 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:13:12.964920 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc0b642-a45f-47e5-99b5-624bed9f457c None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:13:13.062118 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc0b642-a45f-47e5-99b5-624bed9f457c None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:13:13.084169 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:13:13.095203 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ed6c96-5095-4647-8d06-b743e4c3fd5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:13:13.095799 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ed6c96-5095-4647-8d06-b743e4c3fd5e None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:13:13.096077 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ed6c96-5095-4647-8d06-b743e4c3fd5e None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:13:13.096346 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ed6c96-5095-4647-8d06-b743e4c3fd5e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:13:13.160214 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ed6c96-5095-4647-8d06-b743e4c3fd5e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:13:13.161676 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ed6c96-5095-4647-8d06-b743e4c3fd5e None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:13:13.255271 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 595/1183] 10.222.0.15 () {58 vars in 1282 bytes} [Thu Sep 15 12:13:12 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 15 12:13:13.261193 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b29222e-8010-4222-815b-d370f9405f96 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:13:13.275889 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:13:13.357032 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b29222e-8010-4222-815b-d370f9405f96 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:13:13.392189 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982f9df6-241d-4584-b291-acc7fd85dd6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:13:13.392840 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982f9df6-241d-4584-b291-acc7fd85dd6e None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:13:13.393122 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982f9df6-241d-4584-b291-acc7fd85dd6e None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:13:13.393393 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982f9df6-241d-4584-b291-acc7fd85dd6e None admin] PATH_INFO: `/v3/projects/bc6ae5bb670e4ff38b71ee7db6c6e788` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:13:13.394488 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982f9df6-241d-4584-b291-acc7fd85dd6e None admin] RBAC: Authorizing `identity:get_project(project_id=bc6ae5bb670e4ff38b71ee7db6c6e788)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:13:13.395899 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982f9df6-241d-4584-b291-acc7fd85dd6e None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:13:13.397525 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 589/1184] 10.222.0.15 () {56 vars in 1186 bytes} [Thu Sep 15 12:13:13 2022] GET /identity/v3/projects/bc6ae5bb670e4ff38b71ee7db6c6e788 => generated 361 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:13:13.403597 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ed6c96-5095-4647-8d06-b743e4c3fd5e None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:13:13.426981 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:13:13.498401 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ed6c96-5095-4647-8d06-b743e4c3fd5e None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:13:13.531810 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6813e4c7-50a7-48ab-96fa-0eda9860e77a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:13:13.532396 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6813e4c7-50a7-48ab-96fa-0eda9860e77a None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:13:13.532668 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6813e4c7-50a7-48ab-96fa-0eda9860e77a None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:13:13.532968 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6813e4c7-50a7-48ab-96fa-0eda9860e77a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:13:13.533674 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6813e4c7-50a7-48ab-96fa-0eda9860e77a None admin] RBAC: Adding query filter params (role.id=9c769b5c82884766b2c510b1f0f1364c, scope.project.id=bc6ae5bb670e4ff38b71ee7db6c6e788, user.id=e50b1637d55b4d45bada99bf95cfdff7) {{(pid=81449) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:13:13.533972 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6813e4c7-50a7-48ab-96fa-0eda9860e77a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:13:13.535375 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6813e4c7-50a7-48ab-96fa-0eda9860e77a None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:13:13.540542 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 596/1185] 10.222.0.15 () {56 vars in 1374 bytes} [Thu Sep 15 12:13:13 2022] GET /identity/v3/role_assignments?user.id=e50b1637d55b4d45bada99bf95cfdff7&scope.project.id=bc6ae5bb670e4ff38b71ee7db6c6e788&role.id=9c769b5c82884766b2c510b1f0f1364c => generated 259 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:13:15.626339 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1887eadd-d003-41f6-bdec-97635e5a561c None None] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:13:15.626847 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1887eadd-d003-41f6-bdec-97635e5a561c None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:13:15.626938 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1887eadd-d003-41f6-bdec-97635e5a561c None None] PATH_INFO: `/` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:13:15.628202 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 590/1186] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 12:13:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:13:15.633383 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051307de-6e77-45d6-bb6d-642933df0f83 None None] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:13:15.633778 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051307de-6e77-45d6-bb6d-642933df0f83 None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:13:15.633924 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051307de-6e77-45d6-bb6d-642933df0f83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:13:15.652212 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.auth.core [None req-051307de-6e77-45d6-bb6d-642933df0f83 None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81449) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:13:15.653420 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-051307de-6e77-45d6-bb6d-642933df0f83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:13:15.716342 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 597/1187] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 12:13:15 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 15 12:13:15.982142 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1887eadd-d003-41f6-bdec-97635e5a561c None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:13:16.060510 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:13:16.080196 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1887eadd-d003-41f6-bdec-97635e5a561c None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:13:16.112349 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20889fee-e126-460a-8090-1ec33473c095 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:13:16.112990 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20889fee-e126-460a-8090-1ec33473c095 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:13:16.113269 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20889fee-e126-460a-8090-1ec33473c095 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:13:16.113537 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20889fee-e126-460a-8090-1ec33473c095 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:13:16.123129 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20889fee-e126-460a-8090-1ec33473c095 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 12:13:16.128438 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application [None req-20889fee-e126-460a-8090-1ec33473c095 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 15 12:13:16.128438 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:13:16.128438 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:13:16.128438 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:13:16.128438 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:13:16.128438 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:13:16.128438 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:13:16.128438 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:13:16.128438 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:13:16.128438 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:13:16.128438 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:13:16.128438 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:13:16.128438 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 12:13:16.128438 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 12:13:16.128438 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 12:13:16.128438 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 12:13:16.129584 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 12:13:16.129584 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 12:13:16.129584 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 12:13:16.129584 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 12:13:16.129584 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:13:16.129584 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:13:16.129584 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:13:16.129584 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:13:16.129584 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:13:16.129584 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:13:16.129584 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:13:16.129584 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application with Lock( Sep 15 12:13:16.129584 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:13:16.129584 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:13:16.129584 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:13:16.129584 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:13:16.130699 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:13:16.130699 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:13:16.130699 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:13:16.130699 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:13:16.130699 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 12:13:16.130699 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 12:13:16.130699 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 12:13:16.130699 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 12:13:16.130699 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 12:13:16.130699 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 12:13:16.130699 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 15 12:13:16.130699 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 15 12:13:16.132900 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 591/1188] 10.222.0.15 () {56 vars in 1096 bytes} [Thu Sep 15 12:13:15 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:13:16.135304 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051307de-6e77-45d6-bb6d-642933df0f83 None None] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:13:16.152888 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:13:16.229643 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051307de-6e77-45d6-bb6d-642933df0f83 None None] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:13:16.265377 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47cf217e-60d7-48dc-875c-e178dd0fe21b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:13:16.265983 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cf217e-60d7-48dc-875c-e178dd0fe21b None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:13:16.266262 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cf217e-60d7-48dc-875c-e178dd0fe21b None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:13:16.266533 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cf217e-60d7-48dc-875c-e178dd0fe21b None admin] PATH_INFO: `/v3/roles` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:13:16.267185 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47cf217e-60d7-48dc-875c-e178dd0fe21b None admin] RBAC: Adding query filter params (name=admin) {{(pid=81449) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:13:16.267483 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47cf217e-60d7-48dc-875c-e178dd0fe21b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:13:16.268882 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47cf217e-60d7-48dc-875c-e178dd0fe21b None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:13:16.276721 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 598/1189] 10.222.0.15 () {56 vars in 1099 bytes} [Thu Sep 15 12:13:16 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:13:16.283080 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20889fee-e126-460a-8090-1ec33473c095 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:13:16.378264 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20889fee-e126-460a-8090-1ec33473c095 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:13:16.412267 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5b6af9-5e28-43ca-bd23-f5956b263cf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:13:16.412910 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5b6af9-5e28-43ca-bd23-f5956b263cf2 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:13:16.413186 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5b6af9-5e28-43ca-bd23-f5956b263cf2 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:13:16.413449 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5b6af9-5e28-43ca-bd23-f5956b263cf2 None admin] PATH_INFO: `/v3/users/e50b1637d55b4d45bada99bf95cfdff7` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:13:16.414543 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5b6af9-5e28-43ca-bd23-f5956b263cf2 None admin] RBAC: Authorizing `identity:get_user(user_id=e50b1637d55b4d45bada99bf95cfdff7)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:13:16.415942 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5b6af9-5e28-43ca-bd23-f5956b263cf2 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:13:16.417498 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 592/1190] 10.222.0.15 () {56 vars in 1177 bytes} [Thu Sep 15 12:13:16 2022] GET /identity/v3/users/e50b1637d55b4d45bada99bf95cfdff7 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:13:16.423705 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47cf217e-60d7-48dc-875c-e178dd0fe21b None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:13:16.501516 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:13:16.520061 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47cf217e-60d7-48dc-875c-e178dd0fe21b None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:13:16.552811 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a06e9c2-3899-4da1-9c16-f8b0b3870e5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:13:16.553406 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a06e9c2-3899-4da1-9c16-f8b0b3870e5c None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:13:16.553681 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a06e9c2-3899-4da1-9c16-f8b0b3870e5c None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:13:16.553946 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a06e9c2-3899-4da1-9c16-f8b0b3870e5c None admin] PATH_INFO: `/v3/projects/bc6ae5bb670e4ff38b71ee7db6c6e788` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:13:16.555035 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a06e9c2-3899-4da1-9c16-f8b0b3870e5c None admin] RBAC: Authorizing `identity:get_project(project_id=bc6ae5bb670e4ff38b71ee7db6c6e788)` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:13:16.556445 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a06e9c2-3899-4da1-9c16-f8b0b3870e5c None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:13:16.558051 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 599/1191] 10.222.0.15 () {56 vars in 1186 bytes} [Thu Sep 15 12:13:16 2022] GET /identity/v3/projects/bc6ae5bb670e4ff38b71ee7db6c6e788 => generated 361 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:13:16.564438 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5b6af9-5e28-43ca-bd23-f5956b263cf2 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:13:16.575072 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:13:16.661536 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5b6af9-5e28-43ca-bd23-f5956b263cf2 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:13:16.692917 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58fdbdb-405c-476c-9461-180a2244362b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:13:16.693561 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58fdbdb-405c-476c-9461-180a2244362b None admin] REQUEST_METHOD: `PUT` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:13:16.693836 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58fdbdb-405c-476c-9461-180a2244362b None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:13:16.694101 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58fdbdb-405c-476c-9461-180a2244362b None admin] PATH_INFO: `/v3/projects/bc6ae5bb670e4ff38b71ee7db6c6e788/users/e50b1637d55b4d45bada99bf95cfdff7/roles/9c769b5c82884766b2c510b1f0f1364c` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:13:16.695928 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58fdbdb-405c-476c-9461-180a2244362b None admin] RBAC: Authorizing `identity:create_grant(project_id=bc6ae5bb670e4ff38b71ee7db6c6e788, user_id=e50b1637d55b4d45bada99bf95cfdff7, role_id=9c769b5c82884766b2c510b1f0f1364c)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:13:16.697359 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58fdbdb-405c-476c-9461-180a2244362b None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:13:16.709757 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 593/1192] 10.222.0.15 () {58 vars in 1439 bytes} [Thu Sep 15 12:13:16 2022] PUT /identity/v3/projects/bc6ae5bb670e4ff38b71ee7db6c6e788/users/e50b1637d55b4d45bada99bf95cfdff7/roles/9c769b5c82884766b2c510b1f0f1364c => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:13:18.818188 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33dcb624-c49f-48ad-84cb-3f94110147c6 None None] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:13:18.818722 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33dcb624-c49f-48ad-84cb-3f94110147c6 None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:13:18.818722 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33dcb624-c49f-48ad-84cb-3f94110147c6 None None] PATH_INFO: `/` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:13:18.820044 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 600/1193] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 12:13:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:13:18.825227 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da213b09-1283-419f-a23c-d7afd7b72372 None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:13:18.825627 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da213b09-1283-419f-a23c-d7afd7b72372 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:13:18.825779 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da213b09-1283-419f-a23c-d7afd7b72372 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:13:18.846513 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-da213b09-1283-419f-a23c-d7afd7b72372 None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:13:18.847692 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-da213b09-1283-419f-a23c-d7afd7b72372 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:13:18.910193 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:13:18.916857 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 594/1194] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 12:13:18 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 15 12:13:19.185603 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33dcb624-c49f-48ad-84cb-3f94110147c6 None None] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:13:19.282822 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33dcb624-c49f-48ad-84cb-3f94110147c6 None None] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:13:19.314687 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b635a502-d4b3-4082-97c2-53f29f0ccc42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:13:19.315293 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b635a502-d4b3-4082-97c2-53f29f0ccc42 None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:13:19.315569 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b635a502-d4b3-4082-97c2-53f29f0ccc42 None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:13:19.315836 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b635a502-d4b3-4082-97c2-53f29f0ccc42 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:13:19.325428 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b635a502-d4b3-4082-97c2-53f29f0ccc42 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 12:13:19.330663 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application [None req-b635a502-d4b3-4082-97c2-53f29f0ccc42 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 15 12:13:19.330663 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:13:19.330663 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:13:19.330663 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:13:19.330663 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:13:19.330663 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:13:19.330663 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:13:19.330663 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:13:19.330663 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:13:19.330663 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:13:19.330663 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:13:19.330663 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:13:19.330663 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 12:13:19.330663 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 12:13:19.330663 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 12:13:19.330663 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 12:13:19.331732 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 12:13:19.331732 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 12:13:19.331732 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 12:13:19.331732 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 12:13:19.331732 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:13:19.331732 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:13:19.331732 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:13:19.331732 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:13:19.331732 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:13:19.331732 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:13:19.331732 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:13:19.331732 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application with Lock( Sep 15 12:13:19.331732 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:13:19.331732 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:13:19.331732 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:13:19.331732 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:13:19.332995 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:13:19.332995 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:13:19.332995 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:13:19.332995 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:13:19.332995 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 12:13:19.332995 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 12:13:19.332995 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 12:13:19.332995 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 12:13:19.332995 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 12:13:19.332995 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 12:13:19.332995 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 15 12:13:19.332995 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application Sep 15 12:13:19.332995 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 601/1195] 10.222.0.15 () {56 vars in 1096 bytes} [Thu Sep 15 12:13:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 147 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:13:19.337616 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da213b09-1283-419f-a23c-d7afd7b72372 None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:13:19.430112 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da213b09-1283-419f-a23c-d7afd7b72372 None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:13:19.462748 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ae4d21-11b0-40bd-845b-32749c8caac5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:13:19.463291 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ae4d21-11b0-40bd-845b-32749c8caac5 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:13:19.463566 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ae4d21-11b0-40bd-845b-32749c8caac5 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:13:19.463829 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ae4d21-11b0-40bd-845b-32749c8caac5 None admin] PATH_INFO: `/v3/roles` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:13:19.464472 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ae4d21-11b0-40bd-845b-32749c8caac5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:13:19.464787 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ae4d21-11b0-40bd-845b-32749c8caac5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:13:19.466152 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ae4d21-11b0-40bd-845b-32749c8caac5 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:13:19.473775 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 595/1196] 10.222.0.15 () {56 vars in 1099 bytes} [Thu Sep 15 12:13:19 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:13:19.479793 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b635a502-d4b3-4082-97c2-53f29f0ccc42 None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:13:19.490671 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:13:19.578091 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b635a502-d4b3-4082-97c2-53f29f0ccc42 None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:13:19.613606 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5163522a-52ab-4476-b83b-5a989ad767a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:13:19.614231 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5163522a-52ab-4476-b83b-5a989ad767a6 None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:13:19.614531 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5163522a-52ab-4476-b83b-5a989ad767a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:13:19.614796 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5163522a-52ab-4476-b83b-5a989ad767a6 None admin] PATH_INFO: `/v3/users/e50b1637d55b4d45bada99bf95cfdff7` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:13:19.615885 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5163522a-52ab-4476-b83b-5a989ad767a6 None admin] RBAC: Authorizing `identity:get_user(user_id=e50b1637d55b4d45bada99bf95cfdff7)` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:13:19.617333 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5163522a-52ab-4476-b83b-5a989ad767a6 None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:13:19.618902 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 602/1197] 10.222.0.15 () {56 vars in 1177 bytes} [Thu Sep 15 12:13:19 2022] GET /identity/v3/users/e50b1637d55b4d45bada99bf95cfdff7 => generated 311 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:13:19.625593 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ae4d21-11b0-40bd-845b-32749c8caac5 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:13:19.681778 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:13:19.720216 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ae4d21-11b0-40bd-845b-32749c8caac5 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:13:19.752418 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620aef6d-6751-44b4-a6af-155f820c2774 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:13:19.753028 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620aef6d-6751-44b4-a6af-155f820c2774 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:13:19.753301 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620aef6d-6751-44b4-a6af-155f820c2774 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:13:19.753567 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620aef6d-6751-44b4-a6af-155f820c2774 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:13:19.817846 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620aef6d-6751-44b4-a6af-155f820c2774 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:13:19.819314 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620aef6d-6751-44b4-a6af-155f820c2774 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:13:19.873413 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:13:19.913309 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 596/1198] 10.222.0.15 () {58 vars in 1282 bytes} [Thu Sep 15 12:13:19 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 288 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 15 12:13:19.919157 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5163522a-52ab-4476-b83b-5a989ad767a6 None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:13:19.993193 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:13:20.015615 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5163522a-52ab-4476-b83b-5a989ad767a6 None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:13:20.048415 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f19733dd-2042-420c-971b-be2ca4b7ffcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:13:20.049060 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19733dd-2042-420c-971b-be2ca4b7ffcd None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:13:20.049338 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19733dd-2042-420c-971b-be2ca4b7ffcd None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:13:20.049603 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19733dd-2042-420c-971b-be2ca4b7ffcd None admin] PATH_INFO: `/v3/projects/bc6ae5bb670e4ff38b71ee7db6c6e788` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:13:20.050687 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19733dd-2042-420c-971b-be2ca4b7ffcd None admin] RBAC: Authorizing `identity:get_project(project_id=bc6ae5bb670e4ff38b71ee7db6c6e788)` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:13:20.052094 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f19733dd-2042-420c-971b-be2ca4b7ffcd None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:13:20.053699 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 603/1199] 10.222.0.15 () {56 vars in 1186 bytes} [Thu Sep 15 12:13:19 2022] GET /identity/v3/projects/bc6ae5bb670e4ff38b71ee7db6c6e788 => generated 361 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:13:20.059813 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620aef6d-6751-44b4-a6af-155f820c2774 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:13:20.154299 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620aef6d-6751-44b4-a6af-155f820c2774 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:13:20.187914 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b43d1f-7047-4da0-8a1e-4fc3c6ff5ffb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:13:20.188505 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b43d1f-7047-4da0-8a1e-4fc3c6ff5ffb None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:13:20.188809 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b43d1f-7047-4da0-8a1e-4fc3c6ff5ffb None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:13:20.189084 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b43d1f-7047-4da0-8a1e-4fc3c6ff5ffb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:13:20.189794 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b43d1f-7047-4da0-8a1e-4fc3c6ff5ffb None admin] RBAC: Adding query filter params (role.id=9c769b5c82884766b2c510b1f0f1364c, scope.project.id=bc6ae5bb670e4ff38b71ee7db6c6e788, user.id=e50b1637d55b4d45bada99bf95cfdff7) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:13:20.190099 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b43d1f-7047-4da0-8a1e-4fc3c6ff5ffb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:13:20.191516 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b43d1f-7047-4da0-8a1e-4fc3c6ff5ffb None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:13:20.197155 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 597/1200] 10.222.0.15 () {56 vars in 1374 bytes} [Thu Sep 15 12:13:20 2022] GET /identity/v3/role_assignments?user.id=e50b1637d55b4d45bada99bf95cfdff7&scope.project.id=bc6ae5bb670e4ff38b71ee7db6c6e788&role.id=9c769b5c82884766b2c510b1f0f1364c => generated 608 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:13:30.043844 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=28 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:13:30.194640 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=88 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:14:15.530384 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9dcc76d-eb5d-48fd-8bcf-7ed2af22cdd8 None None] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:14:15.531053 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9dcc76d-eb5d-48fd-8bcf-7ed2af22cdd8 None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:14:15.531053 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9dcc76d-eb5d-48fd-8bcf-7ed2af22cdd8 None None] PATH_INFO: `/` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:14:15.532541 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 604/1201] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 12:14:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:14:15.539097 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009c5c3a-8053-4bc1-a324-1c4422ca8a8a None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:14:15.539638 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009c5c3a-8053-4bc1-a324-1c4422ca8a8a None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:14:15.539764 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009c5c3a-8053-4bc1-a324-1c4422ca8a8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:14:15.560476 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-009c5c3a-8053-4bc1-a324-1c4422ca8a8a None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:14:15.561766 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-009c5c3a-8053-4bc1-a324-1c4422ca8a8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:14:15.670631 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 598/1202] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 12:14:15 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 15 12:14:15.960936 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9dcc76d-eb5d-48fd-8bcf-7ed2af22cdd8 None None] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:14:16.087326 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9dcc76d-eb5d-48fd-8bcf-7ed2af22cdd8 None None] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:14:16.124820 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1e2cd3-3610-4609-b159-199f6c91efae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:14:16.125457 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1e2cd3-3610-4609-b159-199f6c91efae None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:14:16.125740 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1e2cd3-3610-4609-b159-199f6c91efae None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:14:16.126028 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1e2cd3-3610-4609-b159-199f6c91efae None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:14:16.137415 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e1e2cd3-3610-4609-b159-199f6c91efae None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:14:16.138821 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e1e2cd3-3610-4609-b159-199f6c91efae None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:14:16.155054 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application [None req-0e1e2cd3-3610-4609-b159-199f6c91efae None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 12:14:16.155054 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:14:16.155054 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:14:16.155054 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:14:16.155054 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:14:16.155054 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:14:16.155054 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:14:16.155054 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:14:16.155054 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:14:16.155054 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:14:16.155054 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:14:16.155054 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:14:16.155054 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 12:14:16.155054 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 12:14:16.155054 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 12:14:16.155054 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 12:14:16.156446 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:14:16.156446 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:14:16.156446 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:14:16.156446 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:14:16.156446 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:14:16.156446 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:14:16.156446 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:14:16.156446 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application with Lock( Sep 15 12:14:16.156446 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:14:16.156446 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:14:16.156446 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:14:16.156446 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:14:16.156446 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:14:16.156446 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:14:16.156446 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:14:16.156446 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:14:16.156446 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 12:14:16.157853 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 12:14:16.157853 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 12:14:16.157853 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application Sep 15 12:14:16.157853 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 605/1203] 10.222.0.15 () {56 vars in 1108 bytes} [Thu Sep 15 12:14:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:14:16.163298 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009c5c3a-8053-4bc1-a324-1c4422ca8a8a None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:14:16.275526 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009c5c3a-8053-4bc1-a324-1c4422ca8a8a None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:14:16.307356 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:14:16.320789 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5d28aa-c67b-4cec-b435-fa6eba11ec3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:14:16.320789 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5d28aa-c67b-4cec-b435-fa6eba11ec3d None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:14:16.320789 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5d28aa-c67b-4cec-b435-fa6eba11ec3d None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:14:16.320789 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5d28aa-c67b-4cec-b435-fa6eba11ec3d None admin] PATH_INFO: `/v3/domains` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:14:16.321265 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5d28aa-c67b-4cec-b435-fa6eba11ec3d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:14:16.321265 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5d28aa-c67b-4cec-b435-fa6eba11ec3d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:14:16.321265 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5d28aa-c67b-4cec-b435-fa6eba11ec3d None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:14:16.333876 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 599/1204] 10.222.0.15 () {56 vars in 1109 bytes} [Thu Sep 15 12:14:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:14:16.340123 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1e2cd3-3610-4609-b159-199f6c91efae None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:14:16.397654 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:14:16.474043 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1e2cd3-3610-4609-b159-199f6c91efae None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:14:16.514609 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb44c800-c394-415a-aca1-b0bf0a81f76c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:14:16.515259 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb44c800-c394-415a-aca1-b0bf0a81f76c None admin] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:14:16.515553 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb44c800-c394-415a-aca1-b0bf0a81f76c None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:14:16.515880 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb44c800-c394-415a-aca1-b0bf0a81f76c None admin] PATH_INFO: `/v3/users` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:14:16.516540 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb44c800-c394-415a-aca1-b0bf0a81f76c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:14:16.518160 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb44c800-c394-415a-aca1-b0bf0a81f76c None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:14:16.550403 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 606/1205] 10.222.0.15 () {60 vars in 1132 bytes} [Thu Sep 15 12:14:16 2022] POST /identity/v3/users => generated 251 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 12:14:18.879774 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10514e6-f391-4301-bfde-c22c53156f82 None None] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:14:18.880328 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10514e6-f391-4301-bfde-c22c53156f82 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:14:18.880328 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10514e6-f391-4301-bfde-c22c53156f82 None None] PATH_INFO: `/` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:14:18.881846 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 600/1206] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 12:14:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:14:18.887900 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84188ca6-6cf4-4676-a14f-7c03f13200e0 None None] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:14:18.888507 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84188ca6-6cf4-4676-a14f-7c03f13200e0 None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:14:18.888507 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84188ca6-6cf4-4676-a14f-7c03f13200e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:14:18.907360 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.auth.core [None req-84188ca6-6cf4-4676-a14f-7c03f13200e0 None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81449) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:14:18.908537 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-84188ca6-6cf4-4676-a14f-7c03f13200e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:14:18.988792 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 607/1207] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 12:14:18 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 15 12:14:19.259544 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10514e6-f391-4301-bfde-c22c53156f82 None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:14:19.377575 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10514e6-f391-4301-bfde-c22c53156f82 None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:14:19.417278 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1d9712-2d5f-4099-83fc-cf1445ce687b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:14:19.418274 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1d9712-2d5f-4099-83fc-cf1445ce687b None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:14:19.418589 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1d9712-2d5f-4099-83fc-cf1445ce687b None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:14:19.418888 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1d9712-2d5f-4099-83fc-cf1445ce687b None admin] PATH_INFO: `/v3/roles/service` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:14:19.431427 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1d9712-2d5f-4099-83fc-cf1445ce687b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 12:14:19.442331 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application [None req-9a1d9712-2d5f-4099-83fc-cf1445ce687b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 15 12:14:19.442331 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:14:19.442331 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:14:19.442331 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:14:19.442331 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:14:19.442331 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:14:19.442331 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:14:19.442331 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:14:19.442331 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:14:19.442331 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:14:19.442331 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:14:19.442331 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:14:19.442331 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 12:14:19.442331 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 12:14:19.442331 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 12:14:19.442331 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 12:14:19.443542 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 12:14:19.443542 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 12:14:19.443542 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 12:14:19.443542 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 12:14:19.443542 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:14:19.443542 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:14:19.443542 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:14:19.443542 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:14:19.443542 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:14:19.443542 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:14:19.443542 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:14:19.443542 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application with Lock( Sep 15 12:14:19.443542 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:14:19.443542 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:14:19.443542 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:14:19.443542 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:14:19.444841 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:14:19.444841 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:14:19.444841 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:14:19.444841 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:14:19.444841 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 12:14:19.444841 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 12:14:19.444841 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 12:14:19.444841 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 12:14:19.444841 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 12:14:19.444841 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 12:14:19.444841 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 15 12:14:19.444841 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 15 12:14:19.444841 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 601/1208] 10.222.0.15 () {56 vars in 1102 bytes} [Thu Sep 15 12:14:19 2022] GET /identity/v3/roles/service => generated 85 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:14:19.453584 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84188ca6-6cf4-4676-a14f-7c03f13200e0 None None] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:14:19.501498 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:14:19.572074 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84188ca6-6cf4-4676-a14f-7c03f13200e0 None None] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:14:19.613294 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf9f6eb-0eaa-41d5-8851-5229af5e5611 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:14:19.613932 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf9f6eb-0eaa-41d5-8851-5229af5e5611 None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:14:19.614274 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf9f6eb-0eaa-41d5-8851-5229af5e5611 None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:14:19.614535 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf9f6eb-0eaa-41d5-8851-5229af5e5611 None admin] PATH_INFO: `/v3/roles` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:14:19.615253 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf9f6eb-0eaa-41d5-8851-5229af5e5611 None admin] RBAC: Adding query filter params (name=service) {{(pid=81449) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:14:19.615505 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf9f6eb-0eaa-41d5-8851-5229af5e5611 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:14:19.616931 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf9f6eb-0eaa-41d5-8851-5229af5e5611 None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:14:19.627798 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 608/1209] 10.222.0.15 () {56 vars in 1103 bytes} [Thu Sep 15 12:14:19 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:14:19.633836 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1d9712-2d5f-4099-83fc-cf1445ce687b None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:14:19.696634 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:14:19.760416 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1d9712-2d5f-4099-83fc-cf1445ce687b None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:14:19.799433 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18b0ff9-74dc-4fa3-9b39-4cb364e2c751 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:14:19.800074 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18b0ff9-74dc-4fa3-9b39-4cb364e2c751 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:14:19.800348 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18b0ff9-74dc-4fa3-9b39-4cb364e2c751 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:14:19.800613 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18b0ff9-74dc-4fa3-9b39-4cb364e2c751 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:14:19.811846 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a18b0ff9-74dc-4fa3-9b39-4cb364e2c751 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:14:19.813375 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a18b0ff9-74dc-4fa3-9b39-4cb364e2c751 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:14:19.828544 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application [None req-a18b0ff9-74dc-4fa3-9b39-4cb364e2c751 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 12:14:19.828544 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:14:19.828544 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:14:19.828544 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:14:19.828544 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:14:19.828544 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:14:19.828544 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:14:19.828544 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:14:19.828544 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:14:19.828544 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:14:19.828544 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:14:19.828544 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:14:19.828544 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 12:14:19.828544 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 12:14:19.828544 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 12:14:19.828544 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 12:14:19.829687 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:14:19.829687 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:14:19.829687 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:14:19.829687 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:14:19.829687 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:14:19.829687 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:14:19.829687 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:14:19.829687 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application with Lock( Sep 15 12:14:19.829687 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:14:19.829687 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:14:19.829687 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:14:19.829687 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:14:19.829687 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:14:19.829687 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:14:19.829687 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:14:19.829687 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:14:19.829687 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 12:14:19.830823 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 12:14:19.830823 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 12:14:19.830823 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 15 12:14:19.830823 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 602/1210] 10.222.0.15 () {56 vars in 1108 bytes} [Thu Sep 15 12:14:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:14:19.836464 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf9f6eb-0eaa-41d5-8851-5229af5e5611 None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:14:19.944802 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:14:19.949236 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf9f6eb-0eaa-41d5-8851-5229af5e5611 None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:14:19.989824 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f056afc2-0ac0-4114-a009-9634a8b9a215 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:14:19.990410 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f056afc2-0ac0-4114-a009-9634a8b9a215 None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:14:19.991998 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f056afc2-0ac0-4114-a009-9634a8b9a215 None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:14:19.991998 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f056afc2-0ac0-4114-a009-9634a8b9a215 None admin] PATH_INFO: `/v3/domains` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:14:19.991998 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f056afc2-0ac0-4114-a009-9634a8b9a215 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81449) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:14:19.991998 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f056afc2-0ac0-4114-a009-9634a8b9a215 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:14:19.993328 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f056afc2-0ac0-4114-a009-9634a8b9a215 None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:14:20.006895 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 609/1211] 10.222.0.15 () {56 vars in 1109 bytes} [Thu Sep 15 12:14:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:14:20.013220 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18b0ff9-74dc-4fa3-9b39-4cb364e2c751 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:14:20.136748 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18b0ff9-74dc-4fa3-9b39-4cb364e2c751 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:14:20.138981 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:14:20.182954 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1aef5fa-8fc3-41de-9680-a5d33270ea57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:14:20.183936 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1aef5fa-8fc3-41de-9680-a5d33270ea57 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:14:20.184428 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1aef5fa-8fc3-41de-9680-a5d33270ea57 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:14:20.184928 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1aef5fa-8fc3-41de-9680-a5d33270ea57 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:14:20.193888 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1aef5fa-8fc3-41de-9680-a5d33270ea57 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:14:20.195833 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1aef5fa-8fc3-41de-9680-a5d33270ea57 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:14:20.210694 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application [None req-c1aef5fa-8fc3-41de-9680-a5d33270ea57 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 15 12:14:20.210694 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:14:20.210694 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:14:20.210694 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:14:20.210694 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:14:20.210694 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:14:20.210694 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:14:20.210694 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:14:20.210694 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:14:20.210694 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:14:20.210694 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:14:20.210694 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:14:20.210694 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 15 12:14:20.210694 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 15 12:14:20.210694 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 15 12:14:20.210694 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 15 12:14:20.211749 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:14:20.211749 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:14:20.211749 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 15 12:14:20.211749 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 12:14:20.211749 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 15 12:14:20.211749 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 12:14:20.211749 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:14:20.211749 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:14:20.211749 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:14:20.211749 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:14:20.211749 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:14:20.211749 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application with Lock( Sep 15 12:14:20.211749 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:14:20.211749 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:14:20.211749 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:14:20.211749 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:14:20.211749 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:14:20.211749 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:14:20.212779 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:14:20.212779 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:14:20.212779 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 15 12:14:20.212779 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 15 12:14:20.212779 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 15 12:14:20.212779 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 15 12:14:20.212779 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 15 12:14:20.212779 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 15 12:14:20.212779 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 15 12:14:20.212779 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 15 12:14:20.212779 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 603/1212] 10.222.0.15 () {56 vars in 1108 bytes} [Thu Sep 15 12:14:20 2022] GET /identity/v3/users/placement => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:14:20.218881 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f056afc2-0ac0-4114-a009-9634a8b9a215 None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:14:20.331880 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f056afc2-0ac0-4114-a009-9634a8b9a215 None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:14:20.373236 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc52d16-111d-41b7-96b9-81bcecc88b5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:14:20.374157 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc52d16-111d-41b7-96b9-81bcecc88b5d None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:14:20.374510 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc52d16-111d-41b7-96b9-81bcecc88b5d None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:14:20.375018 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc52d16-111d-41b7-96b9-81bcecc88b5d None admin] PATH_INFO: `/v3/users` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:14:20.375933 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc52d16-111d-41b7-96b9-81bcecc88b5d None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81449) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:14:20.376344 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc52d16-111d-41b7-96b9-81bcecc88b5d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:14:20.377982 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc52d16-111d-41b7-96b9-81bcecc88b5d None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:14:20.398449 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 610/1213] 10.222.0.15 () {56 vars in 1143 bytes} [Thu Sep 15 12:14:20 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:14:20.405761 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1aef5fa-8fc3-41de-9680-a5d33270ea57 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:14:20.522358 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1aef5fa-8fc3-41de-9680-a5d33270ea57 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:14:20.562531 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d0cf4c-e258-4ddf-a54b-0ca8958ad2b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:14:20.563144 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d0cf4c-e258-4ddf-a54b-0ca8958ad2b2 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:14:20.563422 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d0cf4c-e258-4ddf-a54b-0ca8958ad2b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:14:20.563684 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d0cf4c-e258-4ddf-a54b-0ca8958ad2b2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:14:20.597268 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:14:20.648630 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d0cf4c-e258-4ddf-a54b-0ca8958ad2b2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:14:20.650176 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d0cf4c-e258-4ddf-a54b-0ca8958ad2b2 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:14:20.762983 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 604/1214] 10.222.0.15 () {58 vars in 1282 bytes} [Thu Sep 15 12:14:20 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 15 12:14:20.768780 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc52d16-111d-41b7-96b9-81bcecc88b5d None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:14:20.788186 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:14:20.880259 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc52d16-111d-41b7-96b9-81bcecc88b5d None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:14:20.916968 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb26f48-6eb7-4e5c-8f3f-c3f97836475e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:14:20.917531 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb26f48-6eb7-4e5c-8f3f-c3f97836475e None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:14:20.917802 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb26f48-6eb7-4e5c-8f3f-c3f97836475e None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:14:20.918696 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb26f48-6eb7-4e5c-8f3f-c3f97836475e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:14:20.929885 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb26f48-6eb7-4e5c-8f3f-c3f97836475e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:14:20.931460 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb26f48-6eb7-4e5c-8f3f-c3f97836475e None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:14:20.949958 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application [None req-5fb26f48-6eb7-4e5c-8f3f-c3f97836475e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 12:14:20.949958 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:14:20.949958 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:14:20.949958 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:14:20.949958 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:14:20.949958 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:14:20.949958 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:14:20.949958 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:14:20.949958 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:14:20.949958 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:14:20.949958 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:14:20.949958 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:14:20.949958 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 12:14:20.949958 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 12:14:20.949958 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 12:14:20.949958 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 12:14:20.951718 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:14:20.951718 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:14:20.951718 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:14:20.951718 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:14:20.951718 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:14:20.951718 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:14:20.951718 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:14:20.951718 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application with Lock( Sep 15 12:14:20.951718 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:14:20.951718 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:14:20.951718 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:14:20.951718 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:14:20.951718 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:14:20.951718 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:14:20.951718 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:14:20.951718 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:14:20.951718 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 12:14:20.952713 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 12:14:20.952713 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 12:14:20.952713 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application Sep 15 12:14:20.952713 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 611/1215] 10.222.0.15 () {56 vars in 1108 bytes} [Thu Sep 15 12:14:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 15 12:14:20.957670 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d0cf4c-e258-4ddf-a54b-0ca8958ad2b2 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:14:21.031865 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:14:21.076800 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d0cf4c-e258-4ddf-a54b-0ca8958ad2b2 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:14:21.113509 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5211acf-1d23-4ce7-bde8-d359bdecc056 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:14:21.114199 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5211acf-1d23-4ce7-bde8-d359bdecc056 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:14:21.114471 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5211acf-1d23-4ce7-bde8-d359bdecc056 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:14:21.114827 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5211acf-1d23-4ce7-bde8-d359bdecc056 None admin] PATH_INFO: `/v3/domains` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:14:21.115592 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5211acf-1d23-4ce7-bde8-d359bdecc056 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:14:21.115962 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5211acf-1d23-4ce7-bde8-d359bdecc056 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:14:21.117451 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5211acf-1d23-4ce7-bde8-d359bdecc056 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:14:21.130953 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 605/1216] 10.222.0.15 () {56 vars in 1109 bytes} [Thu Sep 15 12:14:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:14:21.140320 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb26f48-6eb7-4e5c-8f3f-c3f97836475e None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:14:21.216939 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:14:21.257165 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb26f48-6eb7-4e5c-8f3f-c3f97836475e None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:14:21.301943 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94db0a2-96d8-48f8-aa66-a05b1471a710 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:14:21.302696 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94db0a2-96d8-48f8-aa66-a05b1471a710 None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:14:21.303006 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94db0a2-96d8-48f8-aa66-a05b1471a710 None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:14:21.303332 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94db0a2-96d8-48f8-aa66-a05b1471a710 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:14:21.313550 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94db0a2-96d8-48f8-aa66-a05b1471a710 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:14:21.314988 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f94db0a2-96d8-48f8-aa66-a05b1471a710 None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:14:21.331197 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application [None req-f94db0a2-96d8-48f8-aa66-a05b1471a710 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 12:14:21.331197 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:14:21.331197 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:14:21.331197 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:14:21.331197 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:14:21.331197 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:14:21.331197 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:14:21.331197 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:14:21.331197 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:14:21.331197 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:14:21.331197 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:14:21.331197 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:14:21.331197 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 15 12:14:21.331197 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 15 12:14:21.331197 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 15 12:14:21.332405 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 15 12:14:21.332405 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:14:21.332405 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:14:21.332405 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:14:21.332405 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:14:21.332405 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:14:21.332405 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:14:21.332405 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:14:21.332405 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application with Lock( Sep 15 12:14:21.332405 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:14:21.332405 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:14:21.332405 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:14:21.332405 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:14:21.332405 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:14:21.332405 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:14:21.332405 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:14:21.332405 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:14:21.333669 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 15 12:14:21.333669 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 15 12:14:21.333669 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 15 12:14:21.333669 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 15 12:14:21.333669 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 15 12:14:21.333669 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 15 12:14:21.333669 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 12:14:21.333669 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application Sep 15 12:14:21.333669 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 612/1217] 10.222.0.15 () {56 vars in 1111 bytes} [Thu Sep 15 12:14:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:14:21.338918 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5211acf-1d23-4ce7-bde8-d359bdecc056 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:14:21.453931 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5211acf-1d23-4ce7-bde8-d359bdecc056 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:14:21.478779 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:14:21.491971 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f618b80-a83e-4fd1-8a46-d30d17a438a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:14:21.492737 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f618b80-a83e-4fd1-8a46-d30d17a438a3 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:14:21.493084 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f618b80-a83e-4fd1-8a46-d30d17a438a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:14:21.493255 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f618b80-a83e-4fd1-8a46-d30d17a438a3 None admin] PATH_INFO: `/v3/projects` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:14:21.494030 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f618b80-a83e-4fd1-8a46-d30d17a438a3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:14:21.494337 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f618b80-a83e-4fd1-8a46-d30d17a438a3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:14:21.496391 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f618b80-a83e-4fd1-8a46-d30d17a438a3 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:14:21.509680 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 606/1218] 10.222.0.15 () {56 vars in 1148 bytes} [Thu Sep 15 12:14:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:14:21.516191 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94db0a2-96d8-48f8-aa66-a05b1471a710 None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:14:21.632068 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94db0a2-96d8-48f8-aa66-a05b1471a710 None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:14:21.664544 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:14:21.672646 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5bdf8a3-9a57-4fb8-8991-85d0da7c0558 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:14:21.673348 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bdf8a3-9a57-4fb8-8991-85d0da7c0558 None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:14:21.673579 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bdf8a3-9a57-4fb8-8991-85d0da7c0558 None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:14:21.673873 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bdf8a3-9a57-4fb8-8991-85d0da7c0558 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:14:21.674652 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5bdf8a3-9a57-4fb8-8991-85d0da7c0558 None admin] RBAC: Adding query filter params (role.id=e1cecc29e028497f9cdaa04268c6a6ec, scope.project.id=540a32ef88764e8c8d9201ae22f6ddaf, user.id=51f64a058bd5440cb5268f228aa6e392) {{(pid=81449) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:14:21.674959 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5bdf8a3-9a57-4fb8-8991-85d0da7c0558 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:14:21.676560 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5bdf8a3-9a57-4fb8-8991-85d0da7c0558 None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:14:21.690207 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 613/1219] 10.222.0.15 () {56 vars in 1374 bytes} [Thu Sep 15 12:14:21 2022] GET /identity/v3/role_assignments?user.id=51f64a058bd5440cb5268f228aa6e392&scope.project.id=540a32ef88764e8c8d9201ae22f6ddaf&role.id=e1cecc29e028497f9cdaa04268c6a6ec => generated 259 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:14:23.882789 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3498822-9399-4bd4-843d-6bfd051bfd7b None None] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:14:23.883368 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3498822-9399-4bd4-843d-6bfd051bfd7b None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:14:23.883368 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3498822-9399-4bd4-843d-6bfd051bfd7b None None] PATH_INFO: `/` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:14:23.884730 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 607/1220] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 12:14:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:14:23.890695 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fb2f21-ed52-412b-a9fa-6784a5987882 None None] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:14:23.891146 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fb2f21-ed52-412b-a9fa-6784a5987882 None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:14:23.891146 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fb2f21-ed52-412b-a9fa-6784a5987882 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:14:23.909459 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.auth.core [None req-a7fb2f21-ed52-412b-a9fa-6784a5987882 None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81449) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:14:23.910642 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-a7fb2f21-ed52-412b-a9fa-6784a5987882 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:14:23.988139 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 614/1221] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 12:14:23 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Sep 15 12:14:24.274536 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3498822-9399-4bd4-843d-6bfd051bfd7b None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:14:24.383758 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3498822-9399-4bd4-843d-6bfd051bfd7b None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:14:24.424571 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca03e99-c594-4b87-be98-12a064ffc805 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:14:24.425240 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca03e99-c594-4b87-be98-12a064ffc805 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:14:24.425518 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca03e99-c594-4b87-be98-12a064ffc805 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:14:24.425782 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca03e99-c594-4b87-be98-12a064ffc805 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:14:24.440628 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca03e99-c594-4b87-be98-12a064ffc805 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 12:14:24.445383 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application [None req-9ca03e99-c594-4b87-be98-12a064ffc805 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 15 12:14:24.445383 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:14:24.445383 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:14:24.445383 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:14:24.445383 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:14:24.445383 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:14:24.445383 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:14:24.445383 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:14:24.445383 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:14:24.445383 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:14:24.445383 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:14:24.445383 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:14:24.445383 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 12:14:24.445383 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 12:14:24.445383 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 12:14:24.445383 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 12:14:24.446687 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 12:14:24.446687 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 12:14:24.446687 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 12:14:24.446687 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 12:14:24.446687 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:14:24.446687 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:14:24.446687 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:14:24.446687 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:14:24.446687 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:14:24.446687 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:14:24.446687 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:14:24.446687 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application with Lock( Sep 15 12:14:24.446687 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:14:24.446687 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:14:24.446687 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:14:24.446687 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:14:24.447726 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:14:24.447726 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:14:24.447726 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:14:24.447726 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:14:24.447726 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 12:14:24.447726 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 12:14:24.447726 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 12:14:24.447726 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 12:14:24.447726 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 12:14:24.447726 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 12:14:24.447726 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 15 12:14:24.447726 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 15 12:14:24.447726 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 608/1222] 10.222.0.15 () {56 vars in 1102 bytes} [Thu Sep 15 12:14:24 2022] GET /identity/v3/roles/service => generated 85 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:14:24.452618 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7fb2f21-ed52-412b-a9fa-6784a5987882 None None] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:14:24.563617 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7fb2f21-ed52-412b-a9fa-6784a5987882 None None] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:14:24.590201 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:14:24.603948 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347b1645-c1c6-4d95-8bdb-aa29d512d764 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:14:24.605266 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347b1645-c1c6-4d95-8bdb-aa29d512d764 None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:14:24.605600 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347b1645-c1c6-4d95-8bdb-aa29d512d764 None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:14:24.605934 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347b1645-c1c6-4d95-8bdb-aa29d512d764 None admin] PATH_INFO: `/v3/roles` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:14:24.606720 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347b1645-c1c6-4d95-8bdb-aa29d512d764 None admin] RBAC: Adding query filter params (name=service) {{(pid=81449) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:14:24.607021 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347b1645-c1c6-4d95-8bdb-aa29d512d764 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:14:24.608941 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347b1645-c1c6-4d95-8bdb-aa29d512d764 None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:14:24.619927 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 615/1223] 10.222.0.15 () {56 vars in 1103 bytes} [Thu Sep 15 12:14:24 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:14:24.626234 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca03e99-c594-4b87-be98-12a064ffc805 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:14:24.671246 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:14:24.735041 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca03e99-c594-4b87-be98-12a064ffc805 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:14:24.770323 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab4678a-fa18-4c9c-a273-ee98436b2e70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:14:24.770984 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab4678a-fa18-4c9c-a273-ee98436b2e70 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:14:24.771261 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab4678a-fa18-4c9c-a273-ee98436b2e70 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:14:24.771535 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab4678a-fa18-4c9c-a273-ee98436b2e70 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:14:24.782165 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab4678a-fa18-4c9c-a273-ee98436b2e70 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:14:24.783607 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab4678a-fa18-4c9c-a273-ee98436b2e70 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:14:24.798710 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application [None req-6ab4678a-fa18-4c9c-a273-ee98436b2e70 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 12:14:24.798710 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:14:24.798710 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:14:24.798710 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:14:24.798710 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:14:24.798710 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:14:24.798710 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:14:24.798710 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:14:24.798710 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:14:24.798710 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:14:24.798710 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:14:24.798710 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:14:24.798710 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 12:14:24.798710 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 12:14:24.798710 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 12:14:24.798710 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 12:14:24.799919 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:14:24.799919 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:14:24.799919 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:14:24.799919 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:14:24.799919 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:14:24.799919 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:14:24.799919 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:14:24.799919 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application with Lock( Sep 15 12:14:24.799919 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:14:24.799919 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:14:24.799919 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:14:24.799919 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:14:24.799919 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:14:24.799919 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:14:24.799919 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:14:24.799919 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:14:24.799919 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 12:14:24.801034 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 12:14:24.801034 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 12:14:24.801034 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 15 12:14:24.801034 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 609/1224] 10.222.0.15 () {56 vars in 1108 bytes} [Thu Sep 15 12:14:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 15 12:14:24.805641 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347b1645-c1c6-4d95-8bdb-aa29d512d764 None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:14:24.914368 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347b1645-c1c6-4d95-8bdb-aa29d512d764 None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:14:24.951287 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e672413-8883-4e88-bacb-1ccbaa09e89c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:14:24.951948 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e672413-8883-4e88-bacb-1ccbaa09e89c None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:14:24.952268 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e672413-8883-4e88-bacb-1ccbaa09e89c None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:14:24.952626 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e672413-8883-4e88-bacb-1ccbaa09e89c None admin] PATH_INFO: `/v3/domains` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:14:24.955836 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e672413-8883-4e88-bacb-1ccbaa09e89c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81449) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:14:24.955836 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e672413-8883-4e88-bacb-1ccbaa09e89c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:14:24.955836 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e672413-8883-4e88-bacb-1ccbaa09e89c None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:14:24.968630 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 616/1225] 10.222.0.15 () {56 vars in 1109 bytes} [Thu Sep 15 12:14:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:14:24.974721 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab4678a-fa18-4c9c-a273-ee98436b2e70 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:14:25.082122 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:14:25.090180 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab4678a-fa18-4c9c-a273-ee98436b2e70 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:14:25.126304 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e84c21-c480-462e-9e6d-e2525f98292a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:14:25.127035 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e84c21-c480-462e-9e6d-e2525f98292a None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:14:25.127350 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e84c21-c480-462e-9e6d-e2525f98292a None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:14:25.127622 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e84c21-c480-462e-9e6d-e2525f98292a None admin] PATH_INFO: `/v3/users/placement` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:14:25.136069 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e84c21-c480-462e-9e6d-e2525f98292a None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:14:25.137692 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e84c21-c480-462e-9e6d-e2525f98292a None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:14:25.150508 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application [None req-13e84c21-c480-462e-9e6d-e2525f98292a None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 15 12:14:25.150508 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:14:25.150508 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:14:25.150508 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:14:25.150508 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:14:25.150508 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:14:25.150508 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:14:25.150508 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:14:25.150508 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:14:25.150508 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:14:25.150508 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:14:25.150508 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:14:25.150508 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 15 12:14:25.150508 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 15 12:14:25.150508 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 15 12:14:25.150508 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 15 12:14:25.152268 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:14:25.152268 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:14:25.152268 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 15 12:14:25.152268 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 12:14:25.152268 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 15 12:14:25.152268 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 12:14:25.152268 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:14:25.152268 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:14:25.152268 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:14:25.152268 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:14:25.152268 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:14:25.152268 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application with Lock( Sep 15 12:14:25.152268 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:14:25.152268 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:14:25.152268 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:14:25.152268 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:14:25.152268 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:14:25.152268 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:14:25.153443 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:14:25.153443 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:14:25.153443 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 15 12:14:25.153443 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 15 12:14:25.153443 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 15 12:14:25.153443 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 15 12:14:25.153443 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 15 12:14:25.153443 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 15 12:14:25.153443 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 15 12:14:25.153443 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 15 12:14:25.153443 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 610/1226] 10.222.0.15 () {56 vars in 1108 bytes} [Thu Sep 15 12:14:24 2022] GET /identity/v3/users/placement => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:14:25.158082 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e672413-8883-4e88-bacb-1ccbaa09e89c None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:14:25.208699 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:14:25.266240 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e672413-8883-4e88-bacb-1ccbaa09e89c None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:14:25.301875 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c0c4fa-e63d-4d53-a46a-87fac152f8f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:14:25.302487 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c0c4fa-e63d-4d53-a46a-87fac152f8f5 None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:14:25.302807 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c0c4fa-e63d-4d53-a46a-87fac152f8f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:14:25.303105 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c0c4fa-e63d-4d53-a46a-87fac152f8f5 None admin] PATH_INFO: `/v3/users` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:14:25.303825 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c0c4fa-e63d-4d53-a46a-87fac152f8f5 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81449) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:14:25.304127 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c0c4fa-e63d-4d53-a46a-87fac152f8f5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:14:25.305568 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c0c4fa-e63d-4d53-a46a-87fac152f8f5 None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:14:25.323611 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 617/1227] 10.222.0.15 () {56 vars in 1143 bytes} [Thu Sep 15 12:14:25 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:14:25.329439 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e84c21-c480-462e-9e6d-e2525f98292a None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:14:25.437440 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e84c21-c480-462e-9e6d-e2525f98292a None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:14:25.477867 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d296b11-24d4-4faa-95de-45629f77daf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:14:25.478434 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d296b11-24d4-4faa-95de-45629f77daf5 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:14:25.478711 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d296b11-24d4-4faa-95de-45629f77daf5 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:14:25.479002 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d296b11-24d4-4faa-95de-45629f77daf5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:14:25.494123 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d296b11-24d4-4faa-95de-45629f77daf5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:14:25.496091 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d296b11-24d4-4faa-95de-45629f77daf5 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:14:25.512482 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application [None req-9d296b11-24d4-4faa-95de-45629f77daf5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 12:14:25.512482 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:14:25.512482 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:14:25.512482 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:14:25.512482 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:14:25.512482 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:14:25.512482 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:14:25.512482 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:14:25.512482 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:14:25.512482 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:14:25.512482 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:14:25.512482 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:14:25.512482 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 12:14:25.512482 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 12:14:25.512482 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 12:14:25.512482 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 12:14:25.514333 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:14:25.514333 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:14:25.514333 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:14:25.514333 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:14:25.514333 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:14:25.514333 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:14:25.514333 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:14:25.514333 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application with Lock( Sep 15 12:14:25.514333 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:14:25.514333 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:14:25.514333 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:14:25.514333 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:14:25.514333 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:14:25.514333 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:14:25.514333 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:14:25.514333 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:14:25.514333 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 12:14:25.516013 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 12:14:25.516013 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 12:14:25.516013 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 15 12:14:25.516013 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 611/1228] 10.222.0.15 () {56 vars in 1108 bytes} [Thu Sep 15 12:14:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:14:25.520577 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c0c4fa-e63d-4d53-a46a-87fac152f8f5 None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:14:25.669177 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c0c4fa-e63d-4d53-a46a-87fac152f8f5 None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:14:25.671883 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:14:25.709912 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c29bcc-445e-4a31-90b4-6f158bfa22a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:14:25.710524 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c29bcc-445e-4a31-90b4-6f158bfa22a4 None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:14:25.710800 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c29bcc-445e-4a31-90b4-6f158bfa22a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:14:25.711072 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c29bcc-445e-4a31-90b4-6f158bfa22a4 None admin] PATH_INFO: `/v3/domains` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:14:25.711708 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c29bcc-445e-4a31-90b4-6f158bfa22a4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81449) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:14:25.712008 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c29bcc-445e-4a31-90b4-6f158bfa22a4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:14:25.713431 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c29bcc-445e-4a31-90b4-6f158bfa22a4 None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:14:25.725588 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 618/1229] 10.222.0.15 () {56 vars in 1109 bytes} [Thu Sep 15 12:14:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:14:25.731769 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d296b11-24d4-4faa-95de-45629f77daf5 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:14:25.741525 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:14:25.839418 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d296b11-24d4-4faa-95de-45629f77daf5 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:14:25.878238 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5355c474-9f51-4724-8263-08071fdf5e40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:14:25.878921 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5355c474-9f51-4724-8263-08071fdf5e40 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:14:25.879196 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5355c474-9f51-4724-8263-08071fdf5e40 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:14:25.879468 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5355c474-9f51-4724-8263-08071fdf5e40 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:14:25.886189 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5355c474-9f51-4724-8263-08071fdf5e40 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:14:25.887666 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5355c474-9f51-4724-8263-08071fdf5e40 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:14:25.900152 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application [None req-5355c474-9f51-4724-8263-08071fdf5e40 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 12:14:25.900152 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:14:25.900152 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:14:25.900152 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:14:25.900152 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:14:25.900152 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:14:25.900152 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:14:25.900152 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:14:25.900152 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:14:25.900152 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:14:25.900152 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:14:25.900152 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:14:25.900152 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 15 12:14:25.900152 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 15 12:14:25.900152 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 15 12:14:25.901844 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 15 12:14:25.901844 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:14:25.901844 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:14:25.901844 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:14:25.901844 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:14:25.901844 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:14:25.901844 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:14:25.901844 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:14:25.901844 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application with Lock( Sep 15 12:14:25.901844 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:14:25.901844 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:14:25.901844 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:14:25.901844 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:14:25.901844 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:14:25.901844 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:14:25.901844 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:14:25.901844 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:14:25.903094 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 15 12:14:25.903094 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 15 12:14:25.903094 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 15 12:14:25.903094 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 15 12:14:25.903094 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 15 12:14:25.903094 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 15 12:14:25.903094 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 12:14:25.903094 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 15 12:14:25.903094 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 612/1230] 10.222.0.15 () {56 vars in 1111 bytes} [Thu Sep 15 12:14:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 15 12:14:25.909602 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c29bcc-445e-4a31-90b4-6f158bfa22a4 None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:14:26.015129 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c29bcc-445e-4a31-90b4-6f158bfa22a4 None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:14:26.050379 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b00f97-2569-46c0-a582-54e8c3b727eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:14:26.051122 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b00f97-2569-46c0-a582-54e8c3b727eb None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:14:26.051391 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b00f97-2569-46c0-a582-54e8c3b727eb None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:14:26.051652 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b00f97-2569-46c0-a582-54e8c3b727eb None admin] PATH_INFO: `/v3/projects` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:14:26.052359 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b00f97-2569-46c0-a582-54e8c3b727eb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81449) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:14:26.052648 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b00f97-2569-46c0-a582-54e8c3b727eb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:14:26.054167 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b00f97-2569-46c0-a582-54e8c3b727eb None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:14:26.069807 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 619/1231] 10.222.0.15 () {56 vars in 1148 bytes} [Thu Sep 15 12:14:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:14:26.074922 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5355c474-9f51-4724-8263-08071fdf5e40 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:14:26.148087 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:14:26.188531 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5355c474-9f51-4724-8263-08071fdf5e40 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:14:26.224316 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc4f590-bb70-452c-8df0-2029bc396902 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:14:26.225431 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc4f590-bb70-452c-8df0-2029bc396902 None admin] REQUEST_METHOD: `PUT` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:14:26.225718 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc4f590-bb70-452c-8df0-2029bc396902 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:14:26.226061 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc4f590-bb70-452c-8df0-2029bc396902 None admin] PATH_INFO: `/v3/projects/540a32ef88764e8c8d9201ae22f6ddaf/users/51f64a058bd5440cb5268f228aa6e392/roles/e1cecc29e028497f9cdaa04268c6a6ec` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:14:26.241029 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc4f590-bb70-452c-8df0-2029bc396902 None admin] RBAC: Authorizing `identity:create_grant(project_id=540a32ef88764e8c8d9201ae22f6ddaf, user_id=51f64a058bd5440cb5268f228aa6e392, role_id=e1cecc29e028497f9cdaa04268c6a6ec)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:14:26.242650 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc4f590-bb70-452c-8df0-2029bc396902 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:14:26.256449 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 613/1232] 10.222.0.15 () {58 vars in 1439 bytes} [Thu Sep 15 12:14:26 2022] PUT /identity/v3/projects/540a32ef88764e8c8d9201ae22f6ddaf/users/51f64a058bd5440cb5268f228aa6e392/roles/e1cecc29e028497f9cdaa04268c6a6ec => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 12:14:28.478251 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fd46d6-ad4a-4f69-95d1-28126b72a94e None None] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:14:28.478818 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fd46d6-ad4a-4f69-95d1-28126b72a94e None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:14:28.478818 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fd46d6-ad4a-4f69-95d1-28126b72a94e None None] PATH_INFO: `/` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:14:28.480386 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 620/1233] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 12:14:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 15 12:14:28.486052 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03550d8a-ac3a-42ed-bc70-13de0fe92b11 None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:14:28.486476 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03550d8a-ac3a-42ed-bc70-13de0fe92b11 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:14:28.486591 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03550d8a-ac3a-42ed-bc70-13de0fe92b11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:14:28.506555 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-03550d8a-ac3a-42ed-bc70-13de0fe92b11 None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:14:28.507819 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-03550d8a-ac3a-42ed-bc70-13de0fe92b11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:14:28.587608 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 614/1234] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 12:14:28 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 15 12:14:28.907201 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46fd46d6-ad4a-4f69-95d1-28126b72a94e None None] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:14:28.938106 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:14:29.030963 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46fd46d6-ad4a-4f69-95d1-28126b72a94e None None] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:14:29.074959 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e3ca31-893c-42d6-af55-f672e693c5fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:14:29.075601 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e3ca31-893c-42d6-af55-f672e693c5fc None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:14:29.075948 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e3ca31-893c-42d6-af55-f672e693c5fc None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:14:29.076136 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e3ca31-893c-42d6-af55-f672e693c5fc None admin] PATH_INFO: `/v3/roles/service` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:14:29.089524 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6e3ca31-893c-42d6-af55-f672e693c5fc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 12:14:29.094863 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application [None req-c6e3ca31-893c-42d6-af55-f672e693c5fc None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Sep 15 12:14:29.094863 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:14:29.094863 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:14:29.094863 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:14:29.094863 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:14:29.094863 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:14:29.094863 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:14:29.094863 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:14:29.094863 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:14:29.094863 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:14:29.094863 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:14:29.094863 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:14:29.094863 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 12:14:29.094863 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 12:14:29.094863 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 12:14:29.094863 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 12:14:29.096057 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 12:14:29.096057 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 12:14:29.096057 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 12:14:29.096057 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 12:14:29.096057 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:14:29.096057 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:14:29.096057 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:14:29.096057 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:14:29.096057 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:14:29.096057 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:14:29.096057 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:14:29.096057 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application with Lock( Sep 15 12:14:29.096057 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:14:29.096057 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:14:29.096057 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:14:29.096057 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:14:29.097077 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:14:29.097077 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:14:29.097077 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:14:29.097077 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:14:29.097077 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 12:14:29.097077 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 12:14:29.097077 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 12:14:29.097077 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 12:14:29.097077 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 12:14:29.097077 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 12:14:29.097077 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Sep 15 12:14:29.097077 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application Sep 15 12:14:29.097077 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 621/1235] 10.222.0.15 () {56 vars in 1102 bytes} [Thu Sep 15 12:14:28 2022] GET /identity/v3/roles/service => generated 85 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 15 12:14:29.103002 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03550d8a-ac3a-42ed-bc70-13de0fe92b11 None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:14:29.159084 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:14:29.219637 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03550d8a-ac3a-42ed-bc70-13de0fe92b11 None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:14:29.261311 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c09c91-887d-4911-9181-c433d594c84e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:14:29.261923 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c09c91-887d-4911-9181-c433d594c84e None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:14:29.262201 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c09c91-887d-4911-9181-c433d594c84e None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:14:29.262472 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c09c91-887d-4911-9181-c433d594c84e None admin] PATH_INFO: `/v3/roles` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:14:29.263125 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c09c91-887d-4911-9181-c433d594c84e None admin] RBAC: Adding query filter params (name=service) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:14:29.263427 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c09c91-887d-4911-9181-c433d594c84e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:14:29.264792 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c09c91-887d-4911-9181-c433d594c84e None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:14:29.272728 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 615/1236] 10.222.0.15 () {56 vars in 1103 bytes} [Thu Sep 15 12:14:29 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:14:29.279431 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e3ca31-893c-42d6-af55-f672e693c5fc None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:14:29.376433 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:14:29.393252 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6e3ca31-893c-42d6-af55-f672e693c5fc None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:14:29.427424 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f171b74-c40a-4ae2-8d01-d9a97d3577ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:14:29.427975 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f171b74-c40a-4ae2-8d01-d9a97d3577ee None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:14:29.428249 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f171b74-c40a-4ae2-8d01-d9a97d3577ee None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:14:29.428515 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f171b74-c40a-4ae2-8d01-d9a97d3577ee None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:14:29.439522 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f171b74-c40a-4ae2-8d01-d9a97d3577ee None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:14:29.441016 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f171b74-c40a-4ae2-8d01-d9a97d3577ee None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:14:29.459098 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application [None req-1f171b74-c40a-4ae2-8d01-d9a97d3577ee None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 12:14:29.459098 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:14:29.459098 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:14:29.459098 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:14:29.459098 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:14:29.459098 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:14:29.459098 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:14:29.459098 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:14:29.459098 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:14:29.459098 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:14:29.459098 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:14:29.459098 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:14:29.459098 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 12:14:29.459098 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 12:14:29.459098 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 12:14:29.459098 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 12:14:29.460680 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:14:29.460680 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:14:29.460680 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:14:29.460680 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:14:29.460680 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:14:29.460680 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:14:29.460680 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:14:29.460680 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application with Lock( Sep 15 12:14:29.460680 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:14:29.460680 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:14:29.460680 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:14:29.460680 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:14:29.460680 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:14:29.460680 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:14:29.460680 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:14:29.460680 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:14:29.460680 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 12:14:29.462289 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 12:14:29.462289 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 12:14:29.462289 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application Sep 15 12:14:29.462289 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 622/1237] 10.222.0.15 () {56 vars in 1108 bytes} [Thu Sep 15 12:14:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:14:29.466728 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c09c91-887d-4911-9181-c433d594c84e None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:14:29.581197 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c09c91-887d-4911-9181-c433d594c84e None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:14:29.592496 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:14:29.625074 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeff525d-c5a9-4b20-b6a8-86e53d601f0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:14:29.625841 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeff525d-c5a9-4b20-b6a8-86e53d601f0f None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:14:29.626175 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeff525d-c5a9-4b20-b6a8-86e53d601f0f None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:14:29.626503 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeff525d-c5a9-4b20-b6a8-86e53d601f0f None admin] PATH_INFO: `/v3/domains` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:14:29.627283 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeff525d-c5a9-4b20-b6a8-86e53d601f0f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:14:29.627724 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeff525d-c5a9-4b20-b6a8-86e53d601f0f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:14:29.629468 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeff525d-c5a9-4b20-b6a8-86e53d601f0f None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:14:29.644252 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 616/1238] 10.222.0.15 () {56 vars in 1109 bytes} [Thu Sep 15 12:14:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:14:29.651257 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f171b74-c40a-4ae2-8d01-d9a97d3577ee None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:14:29.766678 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f171b74-c40a-4ae2-8d01-d9a97d3577ee None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:14:29.802682 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5e23e1-cb8f-4e98-ad55-58f0e708e44a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:14:29.803320 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5e23e1-cb8f-4e98-ad55-58f0e708e44a None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:14:29.803597 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5e23e1-cb8f-4e98-ad55-58f0e708e44a None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:14:29.803866 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5e23e1-cb8f-4e98-ad55-58f0e708e44a None admin] PATH_INFO: `/v3/users/placement` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:14:29.806625 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:14:29.813186 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5e23e1-cb8f-4e98-ad55-58f0e708e44a None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:14:29.815025 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5e23e1-cb8f-4e98-ad55-58f0e708e44a None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:14:29.827418 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application [None req-6c5e23e1-cb8f-4e98-ad55-58f0e708e44a None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 15 12:14:29.827418 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:14:29.827418 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:14:29.827418 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:14:29.827418 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:14:29.827418 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:14:29.827418 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:14:29.827418 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:14:29.827418 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:14:29.827418 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:14:29.827418 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:14:29.827418 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:14:29.827418 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 15 12:14:29.827418 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 15 12:14:29.827418 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 15 12:14:29.827418 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 15 12:14:29.828573 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:14:29.828573 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:14:29.828573 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 15 12:14:29.828573 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 12:14:29.828573 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 15 12:14:29.828573 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 12:14:29.828573 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:14:29.828573 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:14:29.828573 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:14:29.828573 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:14:29.828573 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:14:29.828573 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application with Lock( Sep 15 12:14:29.828573 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:14:29.828573 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:14:29.828573 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:14:29.828573 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:14:29.828573 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:14:29.828573 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:14:29.829684 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:14:29.829684 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:14:29.829684 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 15 12:14:29.829684 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 15 12:14:29.829684 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 15 12:14:29.829684 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 15 12:14:29.829684 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 15 12:14:29.829684 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 15 12:14:29.829684 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 15 12:14:29.829684 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application Sep 15 12:14:29.829684 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 623/1239] 10.222.0.15 () {56 vars in 1108 bytes} [Thu Sep 15 12:14:29 2022] GET /identity/v3/users/placement => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:14:29.835286 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeff525d-c5a9-4b20-b6a8-86e53d601f0f None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:14:29.946932 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeff525d-c5a9-4b20-b6a8-86e53d601f0f None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:14:29.983411 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa7b972-58dc-4370-b126-d12855a91f38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:14:29.984027 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa7b972-58dc-4370-b126-d12855a91f38 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:14:29.984316 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa7b972-58dc-4370-b126-d12855a91f38 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:14:29.984563 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa7b972-58dc-4370-b126-d12855a91f38 None admin] PATH_INFO: `/v3/users` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:14:29.985324 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa7b972-58dc-4370-b126-d12855a91f38 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:14:29.985626 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa7b972-58dc-4370-b126-d12855a91f38 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:14:29.987020 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa7b972-58dc-4370-b126-d12855a91f38 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:14:30.008781 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 617/1240] 10.222.0.15 () {56 vars in 1143 bytes} [Thu Sep 15 12:14:29 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 12:14:30.014087 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5e23e1-cb8f-4e98-ad55-58f0e708e44a None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:14:30.121113 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5e23e1-cb8f-4e98-ad55-58f0e708e44a None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:14:30.155599 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326bd890-9d78-42a6-a004-a794997be938 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:14:30.156320 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326bd890-9d78-42a6-a004-a794997be938 None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:14:30.156634 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326bd890-9d78-42a6-a004-a794997be938 None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:14:30.156969 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326bd890-9d78-42a6-a004-a794997be938 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:14:30.220633 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:14:30.228358 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326bd890-9d78-42a6-a004-a794997be938 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:14:30.230417 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326bd890-9d78-42a6-a004-a794997be938 None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:14:30.333112 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 624/1241] 10.222.0.15 () {58 vars in 1282 bytes} [Thu Sep 15 12:14:30 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 15 12:14:30.338944 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa7b972-58dc-4370-b126-d12855a91f38 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:14:30.370652 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:14:30.447014 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa7b972-58dc-4370-b126-d12855a91f38 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:14:30.482579 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3d81d9-ff4e-4bb2-bceb-49298850a2e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:14:30.483219 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3d81d9-ff4e-4bb2-bceb-49298850a2e5 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:14:30.483507 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3d81d9-ff4e-4bb2-bceb-49298850a2e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:14:30.483762 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3d81d9-ff4e-4bb2-bceb-49298850a2e5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:14:30.493991 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd3d81d9-ff4e-4bb2-bceb-49298850a2e5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:14:30.495993 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd3d81d9-ff4e-4bb2-bceb-49298850a2e5 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:14:30.510579 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application [None req-dd3d81d9-ff4e-4bb2-bceb-49298850a2e5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 12:14:30.510579 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:14:30.510579 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:14:30.510579 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:14:30.510579 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:14:30.510579 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:14:30.510579 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:14:30.510579 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:14:30.510579 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:14:30.510579 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:14:30.510579 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:14:30.510579 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:14:30.510579 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 12:14:30.510579 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 12:14:30.510579 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 12:14:30.510579 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 12:14:30.516395 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:14:30.516395 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:14:30.516395 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:14:30.516395 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:14:30.516395 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:14:30.516395 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:14:30.516395 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:14:30.516395 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application with Lock( Sep 15 12:14:30.516395 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:14:30.516395 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:14:30.516395 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:14:30.516395 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:14:30.516395 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:14:30.516395 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:14:30.516395 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:14:30.516395 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:14:30.516395 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 12:14:30.517370 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 12:14:30.517370 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 12:14:30.517370 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 15 12:14:30.517370 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 618/1242] 10.222.0.15 () {56 vars in 1108 bytes} [Thu Sep 15 12:14:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:14:30.517600 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326bd890-9d78-42a6-a004-a794997be938 None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:14:30.621649 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:14:30.628624 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326bd890-9d78-42a6-a004-a794997be938 None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:14:30.669649 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff35ad6-da94-4733-965a-129beeaa88a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:14:30.670372 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff35ad6-da94-4733-965a-129beeaa88a2 None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:14:30.670585 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff35ad6-da94-4733-965a-129beeaa88a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:14:30.670922 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff35ad6-da94-4733-965a-129beeaa88a2 None admin] PATH_INFO: `/v3/domains` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:14:30.671566 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff35ad6-da94-4733-965a-129beeaa88a2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81449) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:14:30.671855 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff35ad6-da94-4733-965a-129beeaa88a2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:14:30.673376 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff35ad6-da94-4733-965a-129beeaa88a2 None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:14:30.686353 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 625/1243] 10.222.0.15 () {56 vars in 1109 bytes} [Thu Sep 15 12:14:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:14:30.691668 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3d81d9-ff4e-4bb2-bceb-49298850a2e5 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:14:30.779503 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:14:30.807727 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3d81d9-ff4e-4bb2-bceb-49298850a2e5 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:14:30.846831 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce2ebd8-c4cd-4c1a-96c9-f3cfd72e9ab5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:14:30.847470 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce2ebd8-c4cd-4c1a-96c9-f3cfd72e9ab5 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:14:30.847749 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce2ebd8-c4cd-4c1a-96c9-f3cfd72e9ab5 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:14:30.848016 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ce2ebd8-c4cd-4c1a-96c9-f3cfd72e9ab5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:14:30.855719 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce2ebd8-c4cd-4c1a-96c9-f3cfd72e9ab5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:14:30.857419 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ce2ebd8-c4cd-4c1a-96c9-f3cfd72e9ab5 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:14:30.869710 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application [None req-6ce2ebd8-c4cd-4c1a-96c9-f3cfd72e9ab5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 12:14:30.869710 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:14:30.869710 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:14:30.869710 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:14:30.869710 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:14:30.869710 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:14:30.869710 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:14:30.869710 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:14:30.869710 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:14:30.869710 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:14:30.869710 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:14:30.869710 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:14:30.869710 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 15 12:14:30.869710 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 15 12:14:30.869710 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 15 12:14:30.873600 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 15 12:14:30.873600 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:14:30.873600 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:14:30.873600 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:14:30.873600 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:14:30.873600 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:14:30.873600 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:14:30.873600 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:14:30.873600 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application with Lock( Sep 15 12:14:30.873600 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:14:30.873600 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:14:30.873600 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:14:30.873600 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:14:30.873600 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:14:30.873600 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:14:30.873600 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:14:30.873600 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:14:30.874545 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 15 12:14:30.874545 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 15 12:14:30.874545 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 15 12:14:30.874545 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 15 12:14:30.874545 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 15 12:14:30.874545 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 15 12:14:30.874545 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 12:14:30.874545 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 15 12:14:30.874545 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 619/1244] 10.222.0.15 () {56 vars in 1111 bytes} [Thu Sep 15 12:14:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:14:30.876729 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff35ad6-da94-4733-965a-129beeaa88a2 None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:14:30.986541 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff35ad6-da94-4733-965a-129beeaa88a2 None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:14:31.023867 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a6bdc25-7873-4449-a011-4a3c73a9be22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:14:31.024505 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6bdc25-7873-4449-a011-4a3c73a9be22 None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:14:31.024796 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6bdc25-7873-4449-a011-4a3c73a9be22 None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:14:31.025075 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a6bdc25-7873-4449-a011-4a3c73a9be22 None admin] PATH_INFO: `/v3/projects` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:14:31.026403 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6bdc25-7873-4449-a011-4a3c73a9be22 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81449) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:14:31.026710 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6bdc25-7873-4449-a011-4a3c73a9be22 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:14:31.028353 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a6bdc25-7873-4449-a011-4a3c73a9be22 None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:14:31.044939 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 626/1245] 10.222.0.15 () {56 vars in 1148 bytes} [Thu Sep 15 12:14:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:14:31.047011 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce2ebd8-c4cd-4c1a-96c9-f3cfd72e9ab5 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:14:31.161322 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ce2ebd8-c4cd-4c1a-96c9-f3cfd72e9ab5 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:14:31.195952 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92842316-eb87-4ebc-b1ab-ab0945e9739f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:14:31.196592 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92842316-eb87-4ebc-b1ab-ab0945e9739f None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:14:31.196893 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92842316-eb87-4ebc-b1ab-ab0945e9739f None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:14:31.197178 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92842316-eb87-4ebc-b1ab-ab0945e9739f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:14:31.198440 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92842316-eb87-4ebc-b1ab-ab0945e9739f None admin] RBAC: Adding query filter params (role.id=e1cecc29e028497f9cdaa04268c6a6ec, scope.project.id=540a32ef88764e8c8d9201ae22f6ddaf, user.id=51f64a058bd5440cb5268f228aa6e392) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:14:31.198750 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92842316-eb87-4ebc-b1ab-ab0945e9739f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:14:31.200389 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92842316-eb87-4ebc-b1ab-ab0945e9739f None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:14:31.207291 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 620/1246] 10.222.0.15 () {56 vars in 1374 bytes} [Thu Sep 15 12:14:31 2022] GET /identity/v3/role_assignments?user.id=51f64a058bd5440cb5268f228aa6e392&scope.project.id=540a32ef88764e8c8d9201ae22f6ddaf&role.id=e1cecc29e028497f9cdaa04268c6a6ec => generated 608 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:14:33.593534 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3adb8243-af71-4965-95c7-b4334ed461ba None None] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:14:33.594099 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3adb8243-af71-4965-95c7-b4334ed461ba None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:14:33.594099 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3adb8243-af71-4965-95c7-b4334ed461ba None None] PATH_INFO: `/` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:14:33.595689 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 627/1247] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 12:14:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:14:33.602400 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed23141a-1597-4ea7-b6a2-b6dc81f1a9f8 None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:14:33.602867 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed23141a-1597-4ea7-b6a2-b6dc81f1a9f8 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:14:33.602959 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed23141a-1597-4ea7-b6a2-b6dc81f1a9f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:14:33.609496 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:14:33.623077 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-ed23141a-1597-4ea7-b6a2-b6dc81f1a9f8 None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:14:33.624400 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-ed23141a-1597-4ea7-b6a2-b6dc81f1a9f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:14:33.701262 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 621/1248] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 12:14:33 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 15 12:14:33.980017 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3adb8243-af71-4965-95c7-b4334ed461ba None None] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:14:33.999720 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:14:34.090253 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3adb8243-af71-4965-95c7-b4334ed461ba None None] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:14:34.130575 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e72350-0d49-4beb-bfb6-c0e7c165662f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:14:34.131352 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e72350-0d49-4beb-bfb6-c0e7c165662f None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:14:34.131726 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e72350-0d49-4beb-bfb6-c0e7c165662f None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:14:34.132070 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e72350-0d49-4beb-bfb6-c0e7c165662f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:14:34.143537 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e72350-0d49-4beb-bfb6-c0e7c165662f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 12:14:34.149097 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application [None req-46e72350-0d49-4beb-bfb6-c0e7c165662f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 15 12:14:34.149097 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:14:34.149097 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:14:34.149097 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:14:34.149097 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:14:34.149097 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:14:34.149097 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:14:34.149097 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:14:34.149097 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:14:34.149097 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:14:34.149097 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:14:34.149097 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:14:34.149097 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 12:14:34.149097 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 12:14:34.149097 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 12:14:34.149097 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 12:14:34.152878 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 12:14:34.152878 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 12:14:34.152878 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 12:14:34.152878 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 12:14:34.152878 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:14:34.152878 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:14:34.152878 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:14:34.152878 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:14:34.152878 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:14:34.152878 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:14:34.152878 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:14:34.152878 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application with Lock( Sep 15 12:14:34.152878 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:14:34.152878 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:14:34.152878 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:14:34.152878 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:14:34.153991 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:14:34.153991 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:14:34.153991 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:14:34.153991 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:14:34.153991 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 12:14:34.153991 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 12:14:34.153991 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 12:14:34.153991 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 12:14:34.153991 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 12:14:34.153991 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 12:14:34.153991 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 15 12:14:34.153991 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application Sep 15 12:14:34.153991 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 628/1249] 10.222.0.15 () {56 vars in 1096 bytes} [Thu Sep 15 12:14:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:14:34.156425 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed23141a-1597-4ea7-b6a2-b6dc81f1a9f8 None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:14:34.264132 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed23141a-1597-4ea7-b6a2-b6dc81f1a9f8 None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:14:34.295364 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:14:34.301116 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7dce848-d24e-404d-9199-e197fde258e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:14:34.301736 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7dce848-d24e-404d-9199-e197fde258e6 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:14:34.302001 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7dce848-d24e-404d-9199-e197fde258e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:14:34.302267 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7dce848-d24e-404d-9199-e197fde258e6 None admin] PATH_INFO: `/v3/roles` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:14:34.302917 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7dce848-d24e-404d-9199-e197fde258e6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:14:34.303227 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7dce848-d24e-404d-9199-e197fde258e6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:14:34.304552 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7dce848-d24e-404d-9199-e197fde258e6 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:14:34.313308 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 622/1250] 10.222.0.15 () {56 vars in 1099 bytes} [Thu Sep 15 12:14:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:14:34.319182 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e72350-0d49-4beb-bfb6-c0e7c165662f None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:14:34.401346 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:14:34.426113 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e72350-0d49-4beb-bfb6-c0e7c165662f None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:14:34.461596 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7188cd70-bd22-4e49-9830-769ba10e0a67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:14:34.462140 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7188cd70-bd22-4e49-9830-769ba10e0a67 None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:14:34.462420 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7188cd70-bd22-4e49-9830-769ba10e0a67 None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:14:34.463201 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7188cd70-bd22-4e49-9830-769ba10e0a67 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:14:34.473150 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7188cd70-bd22-4e49-9830-769ba10e0a67 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:14:34.474856 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7188cd70-bd22-4e49-9830-769ba10e0a67 None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:14:34.490025 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application [None req-7188cd70-bd22-4e49-9830-769ba10e0a67 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 12:14:34.490025 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:14:34.490025 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:14:34.490025 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:14:34.490025 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:14:34.490025 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:14:34.490025 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:14:34.490025 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:14:34.490025 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:14:34.490025 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:14:34.490025 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:14:34.490025 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:14:34.490025 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 12:14:34.490025 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 12:14:34.490025 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 12:14:34.490025 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 12:14:34.491037 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:14:34.491037 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:14:34.491037 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:14:34.491037 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:14:34.491037 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:14:34.491037 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:14:34.491037 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:14:34.491037 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application with Lock( Sep 15 12:14:34.491037 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:14:34.491037 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:14:34.491037 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:14:34.491037 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:14:34.491037 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:14:34.491037 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:14:34.491037 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:14:34.491037 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:14:34.491037 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 12:14:34.491948 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 12:14:34.491948 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 12:14:34.491948 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application Sep 15 12:14:34.491948 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 629/1251] 10.222.0.15 () {56 vars in 1108 bytes} [Thu Sep 15 12:14:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:14:34.497020 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7dce848-d24e-404d-9199-e197fde258e6 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:14:34.605845 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7dce848-d24e-404d-9199-e197fde258e6 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:14:34.648223 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5691af5-8765-408a-93a2-0c8ea6abd78e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:14:34.648991 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5691af5-8765-408a-93a2-0c8ea6abd78e None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:14:34.649317 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5691af5-8765-408a-93a2-0c8ea6abd78e None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:14:34.649594 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5691af5-8765-408a-93a2-0c8ea6abd78e None admin] PATH_INFO: `/v3/domains` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:14:34.650997 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5691af5-8765-408a-93a2-0c8ea6abd78e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:14:34.651412 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5691af5-8765-408a-93a2-0c8ea6abd78e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:14:34.653293 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5691af5-8765-408a-93a2-0c8ea6abd78e None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:14:34.668081 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 623/1252] 10.222.0.15 () {56 vars in 1109 bytes} [Thu Sep 15 12:14:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:14:34.674205 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7188cd70-bd22-4e49-9830-769ba10e0a67 None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:14:34.786874 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7188cd70-bd22-4e49-9830-769ba10e0a67 None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:14:34.814259 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:14:34.823626 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c098aff-bb6f-451c-b187-9653d8702b9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:14:34.824312 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c098aff-bb6f-451c-b187-9653d8702b9c None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:14:34.824655 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c098aff-bb6f-451c-b187-9653d8702b9c None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:14:34.824978 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c098aff-bb6f-451c-b187-9653d8702b9c None admin] PATH_INFO: `/v3/users/placement` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:14:34.833335 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c098aff-bb6f-451c-b187-9653d8702b9c None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:14:34.834885 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c098aff-bb6f-451c-b187-9653d8702b9c None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:14:34.847434 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application [None req-2c098aff-bb6f-451c-b187-9653d8702b9c None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 15 12:14:34.847434 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:14:34.847434 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:14:34.847434 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:14:34.847434 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:14:34.847434 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:14:34.847434 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:14:34.847434 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:14:34.847434 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:14:34.847434 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:14:34.847434 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:14:34.847434 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:14:34.847434 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 15 12:14:34.847434 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 15 12:14:34.847434 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 15 12:14:34.847434 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 15 12:14:34.848567 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:14:34.848567 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:14:34.848567 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 15 12:14:34.848567 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 12:14:34.848567 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 15 12:14:34.848567 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 12:14:34.848567 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:14:34.848567 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:14:34.848567 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:14:34.848567 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:14:34.848567 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:14:34.848567 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application with Lock( Sep 15 12:14:34.848567 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:14:34.848567 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:14:34.848567 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:14:34.848567 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:14:34.848567 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:14:34.848567 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:14:34.850053 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:14:34.850053 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:14:34.850053 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 15 12:14:34.850053 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 15 12:14:34.850053 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 15 12:14:34.850053 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 15 12:14:34.850053 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 15 12:14:34.850053 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 15 12:14:34.850053 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 15 12:14:34.850053 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application Sep 15 12:14:34.850053 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 630/1253] 10.222.0.15 () {56 vars in 1108 bytes} [Thu Sep 15 12:14:34 2022] GET /identity/v3/users/placement => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Sep 15 12:14:34.854814 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5691af5-8765-408a-93a2-0c8ea6abd78e None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:14:34.928191 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:14:34.975749 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5691af5-8765-408a-93a2-0c8ea6abd78e None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:14:35.012556 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d292b1a8-a703-4bad-8992-716ebbffb116 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:14:35.013208 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d292b1a8-a703-4bad-8992-716ebbffb116 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:14:35.013483 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d292b1a8-a703-4bad-8992-716ebbffb116 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:14:35.013780 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d292b1a8-a703-4bad-8992-716ebbffb116 None admin] PATH_INFO: `/v3/users` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:14:35.014518 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d292b1a8-a703-4bad-8992-716ebbffb116 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:14:35.014821 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d292b1a8-a703-4bad-8992-716ebbffb116 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:14:35.016253 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d292b1a8-a703-4bad-8992-716ebbffb116 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:14:35.041410 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 624/1254] 10.222.0.15 () {56 vars in 1143 bytes} [Thu Sep 15 12:14:34 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:14:35.047739 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c098aff-bb6f-451c-b187-9653d8702b9c None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:14:35.152570 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c098aff-bb6f-451c-b187-9653d8702b9c None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:14:35.187783 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69593e2c-cc89-44f4-be3d-dfdad8564a94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:14:35.188406 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69593e2c-cc89-44f4-be3d-dfdad8564a94 None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:14:35.188687 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69593e2c-cc89-44f4-be3d-dfdad8564a94 None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:14:35.189002 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69593e2c-cc89-44f4-be3d-dfdad8564a94 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:14:35.247716 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:14:35.261445 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69593e2c-cc89-44f4-be3d-dfdad8564a94 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:14:35.262915 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69593e2c-cc89-44f4-be3d-dfdad8564a94 None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:14:35.366243 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 631/1255] 10.222.0.15 () {58 vars in 1282 bytes} [Thu Sep 15 12:14:35 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 15 12:14:35.372533 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d292b1a8-a703-4bad-8992-716ebbffb116 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:14:35.481253 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d292b1a8-a703-4bad-8992-716ebbffb116 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:14:35.495885 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:14:35.517511 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8f4fd2-026d-467e-b17b-11a77fb92acb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:14:35.518157 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8f4fd2-026d-467e-b17b-11a77fb92acb None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:14:35.518433 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8f4fd2-026d-467e-b17b-11a77fb92acb None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:14:35.518699 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8f4fd2-026d-467e-b17b-11a77fb92acb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:14:35.528822 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8f4fd2-026d-467e-b17b-11a77fb92acb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:14:35.530278 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8f4fd2-026d-467e-b17b-11a77fb92acb None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:14:35.545564 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application [None req-be8f4fd2-026d-467e-b17b-11a77fb92acb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 12:14:35.545564 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:14:35.545564 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:14:35.545564 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:14:35.545564 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:14:35.545564 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:14:35.545564 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:14:35.545564 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:14:35.545564 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:14:35.545564 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:14:35.545564 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:14:35.545564 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:14:35.545564 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 12:14:35.545564 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 12:14:35.545564 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 12:14:35.545564 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 12:14:35.546590 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:14:35.546590 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:14:35.546590 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:14:35.546590 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:14:35.546590 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:14:35.546590 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:14:35.546590 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:14:35.546590 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application with Lock( Sep 15 12:14:35.546590 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:14:35.546590 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:14:35.546590 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:14:35.546590 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:14:35.546590 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:14:35.546590 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:14:35.546590 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:14:35.546590 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:14:35.546590 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 12:14:35.547514 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 12:14:35.547514 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 12:14:35.547514 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 15 12:14:35.547514 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 625/1256] 10.222.0.15 () {56 vars in 1108 bytes} [Thu Sep 15 12:14:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:14:35.552550 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69593e2c-cc89-44f4-be3d-dfdad8564a94 None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:14:35.643621 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:14:35.657700 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69593e2c-cc89-44f4-be3d-dfdad8564a94 None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:14:35.693518 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe3c377-5207-4cca-a404-cc2914c3bb3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:14:35.695088 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe3c377-5207-4cca-a404-cc2914c3bb3a None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:14:35.695367 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe3c377-5207-4cca-a404-cc2914c3bb3a None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:14:35.695634 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe3c377-5207-4cca-a404-cc2914c3bb3a None admin] PATH_INFO: `/v3/domains` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:14:35.696265 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe3c377-5207-4cca-a404-cc2914c3bb3a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81449) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:14:35.696564 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe3c377-5207-4cca-a404-cc2914c3bb3a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:14:35.698005 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe3c377-5207-4cca-a404-cc2914c3bb3a None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:14:35.708791 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 632/1257] 10.222.0.15 () {56 vars in 1109 bytes} [Thu Sep 15 12:14:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:14:35.714561 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8f4fd2-026d-467e-b17b-11a77fb92acb None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:14:35.811350 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8f4fd2-026d-467e-b17b-11a77fb92acb None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:14:35.847607 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b171e8a5-4599-45b8-a93e-49948723cb9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:14:35.848221 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b171e8a5-4599-45b8-a93e-49948723cb9c None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:14:35.848499 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b171e8a5-4599-45b8-a93e-49948723cb9c None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:14:35.848782 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b171e8a5-4599-45b8-a93e-49948723cb9c None admin] PATH_INFO: `/v3/projects/service` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:14:35.856091 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b171e8a5-4599-45b8-a93e-49948723cb9c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:14:35.857539 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b171e8a5-4599-45b8-a93e-49948723cb9c None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:14:35.868447 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application [None req-b171e8a5-4599-45b8-a93e-49948723cb9c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 12:14:35.868447 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:14:35.868447 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:14:35.868447 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:14:35.868447 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:14:35.868447 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:14:35.868447 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:14:35.868447 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:14:35.868447 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:14:35.868447 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:14:35.868447 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:14:35.868447 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:14:35.868447 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 15 12:14:35.868447 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 15 12:14:35.868447 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 15 12:14:35.869498 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 15 12:14:35.869498 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:14:35.869498 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:14:35.869498 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:14:35.869498 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:14:35.869498 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:14:35.869498 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:14:35.869498 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:14:35.869498 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application with Lock( Sep 15 12:14:35.869498 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:14:35.869498 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:14:35.869498 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:14:35.869498 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:14:35.869498 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:14:35.869498 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:14:35.869498 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:14:35.869498 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:14:35.870549 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 15 12:14:35.870549 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 15 12:14:35.870549 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 15 12:14:35.870549 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 15 12:14:35.870549 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 15 12:14:35.870549 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 15 12:14:35.870549 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 12:14:35.870549 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 15 12:14:35.870549 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 626/1258] 10.222.0.15 () {56 vars in 1111 bytes} [Thu Sep 15 12:14:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:14:35.875520 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe3c377-5207-4cca-a404-cc2914c3bb3a None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:14:35.973441 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe3c377-5207-4cca-a404-cc2914c3bb3a None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:14:36.006857 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6eabec7-497c-4169-98ba-09ddfe0d122c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:14:36.007462 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6eabec7-497c-4169-98ba-09ddfe0d122c None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:14:36.007740 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6eabec7-497c-4169-98ba-09ddfe0d122c None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:14:36.008012 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6eabec7-497c-4169-98ba-09ddfe0d122c None admin] PATH_INFO: `/v3/projects` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:14:36.008685 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6eabec7-497c-4169-98ba-09ddfe0d122c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81449) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:14:36.009019 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6eabec7-497c-4169-98ba-09ddfe0d122c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:14:36.010435 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6eabec7-497c-4169-98ba-09ddfe0d122c None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:14:36.021842 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 633/1259] 10.222.0.15 () {56 vars in 1148 bytes} [Thu Sep 15 12:14:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:14:36.028554 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b171e8a5-4599-45b8-a93e-49948723cb9c None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:14:36.053784 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:14:36.124446 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b171e8a5-4599-45b8-a93e-49948723cb9c None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:14:36.156482 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4715a50f-2977-468d-a51f-bef85b56d902 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:14:36.157112 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4715a50f-2977-468d-a51f-bef85b56d902 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:14:36.157386 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4715a50f-2977-468d-a51f-bef85b56d902 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:14:36.157649 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4715a50f-2977-468d-a51f-bef85b56d902 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:14:36.158355 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4715a50f-2977-468d-a51f-bef85b56d902 None admin] RBAC: Adding query filter params (role.id=9c769b5c82884766b2c510b1f0f1364c, scope.project.id=540a32ef88764e8c8d9201ae22f6ddaf, user.id=51f64a058bd5440cb5268f228aa6e392) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:14:36.158657 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4715a50f-2977-468d-a51f-bef85b56d902 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:14:36.160048 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4715a50f-2977-468d-a51f-bef85b56d902 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:14:36.165152 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 627/1260] 10.222.0.15 () {56 vars in 1374 bytes} [Thu Sep 15 12:14:36 2022] GET /identity/v3/role_assignments?user.id=51f64a058bd5440cb5268f228aa6e392&scope.project.id=540a32ef88764e8c8d9201ae22f6ddaf&role.id=9c769b5c82884766b2c510b1f0f1364c => generated 259 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:14:38.234030 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d4630d-9f8c-4840-8eaa-07cd4973a53d None None] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:14:38.234526 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d4630d-9f8c-4840-8eaa-07cd4973a53d None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:14:38.234526 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d4630d-9f8c-4840-8eaa-07cd4973a53d None None] PATH_INFO: `/` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:14:38.235826 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 634/1261] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 12:14:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:14:38.240818 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54edb310-f3f5-4ec2-a6ac-28129fb96bee None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:14:38.241219 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54edb310-f3f5-4ec2-a6ac-28129fb96bee None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:14:38.241359 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54edb310-f3f5-4ec2-a6ac-28129fb96bee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:14:38.260635 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-54edb310-f3f5-4ec2-a6ac-28129fb96bee None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:14:38.261860 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-54edb310-f3f5-4ec2-a6ac-28129fb96bee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:14:38.324376 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 628/1262] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 12:14:38 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 15 12:14:38.588123 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d4630d-9f8c-4840-8eaa-07cd4973a53d None None] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:14:38.601359 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:14:38.709722 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d4630d-9f8c-4840-8eaa-07cd4973a53d None None] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:14:38.742670 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e965b6-3f24-4667-9575-da69ca6f5473 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:14:38.743290 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e965b6-3f24-4667-9575-da69ca6f5473 None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:14:38.743567 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e965b6-3f24-4667-9575-da69ca6f5473 None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:14:38.744478 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e965b6-3f24-4667-9575-da69ca6f5473 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:14:38.754381 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e965b6-3f24-4667-9575-da69ca6f5473 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 12:14:38.759049 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application [None req-68e965b6-3f24-4667-9575-da69ca6f5473 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 15 12:14:38.759049 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:14:38.759049 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:14:38.759049 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:14:38.759049 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:14:38.759049 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:14:38.759049 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:14:38.759049 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:14:38.759049 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:14:38.759049 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:14:38.759049 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:14:38.759049 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:14:38.759049 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 12:14:38.759049 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 12:14:38.759049 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 12:14:38.759049 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 12:14:38.760162 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 12:14:38.760162 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 12:14:38.760162 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 12:14:38.760162 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 12:14:38.760162 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:14:38.760162 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:14:38.760162 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:14:38.760162 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:14:38.760162 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:14:38.760162 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:14:38.760162 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:14:38.760162 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application with Lock( Sep 15 12:14:38.760162 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:14:38.760162 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:14:38.760162 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:14:38.760162 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:14:38.761395 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:14:38.761395 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:14:38.761395 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:14:38.761395 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:14:38.761395 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 12:14:38.761395 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 12:14:38.761395 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 12:14:38.761395 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 12:14:38.761395 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 12:14:38.761395 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 12:14:38.761395 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 15 12:14:38.761395 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application Sep 15 12:14:38.761395 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 635/1263] 10.222.0.15 () {56 vars in 1096 bytes} [Thu Sep 15 12:14:38 2022] GET /identity/v3/roles/admin => generated 83 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:14:38.766193 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54edb310-f3f5-4ec2-a6ac-28129fb96bee None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:14:38.784130 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:14:38.861048 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54edb310-f3f5-4ec2-a6ac-28129fb96bee None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:14:38.892248 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e17272-6dcb-4659-8674-e388bfe883c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:14:38.892864 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e17272-6dcb-4659-8674-e388bfe883c0 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:14:38.893139 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e17272-6dcb-4659-8674-e388bfe883c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:14:38.893405 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e17272-6dcb-4659-8674-e388bfe883c0 None admin] PATH_INFO: `/v3/roles` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:14:38.894047 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e17272-6dcb-4659-8674-e388bfe883c0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:14:38.894343 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e17272-6dcb-4659-8674-e388bfe883c0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:14:38.895685 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e17272-6dcb-4659-8674-e388bfe883c0 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:14:38.903750 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 629/1264] 10.222.0.15 () {56 vars in 1099 bytes} [Thu Sep 15 12:14:38 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 12:14:38.909349 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e965b6-3f24-4667-9575-da69ca6f5473 None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:14:38.974232 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:14:39.008415 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e965b6-3f24-4667-9575-da69ca6f5473 None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:14:39.041555 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51408039-0fab-4178-bc85-6df0dadeb3cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:14:39.042171 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51408039-0fab-4178-bc85-6df0dadeb3cc None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:14:39.042448 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51408039-0fab-4178-bc85-6df0dadeb3cc None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:14:39.042717 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51408039-0fab-4178-bc85-6df0dadeb3cc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:14:39.052224 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51408039-0fab-4178-bc85-6df0dadeb3cc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:14:39.053690 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51408039-0fab-4178-bc85-6df0dadeb3cc None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:14:39.068406 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application [None req-51408039-0fab-4178-bc85-6df0dadeb3cc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 12:14:39.068406 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:14:39.068406 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:14:39.068406 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:14:39.068406 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:14:39.068406 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:14:39.068406 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:14:39.068406 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:14:39.068406 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:14:39.068406 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:14:39.068406 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:14:39.068406 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:14:39.068406 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 12:14:39.068406 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 12:14:39.068406 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 12:14:39.068406 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 12:14:39.069542 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:14:39.069542 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:14:39.069542 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:14:39.069542 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:14:39.069542 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:14:39.069542 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:14:39.069542 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:14:39.069542 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application with Lock( Sep 15 12:14:39.069542 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:14:39.069542 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:14:39.069542 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:14:39.069542 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:14:39.069542 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:14:39.069542 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:14:39.069542 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:14:39.069542 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:14:39.069542 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 12:14:39.070787 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 12:14:39.070787 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 12:14:39.070787 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application Sep 15 12:14:39.070787 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 636/1265] 10.222.0.15 () {56 vars in 1108 bytes} [Thu Sep 15 12:14:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:14:39.075278 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e17272-6dcb-4659-8674-e388bfe883c0 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:14:39.165041 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:14:39.183679 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e17272-6dcb-4659-8674-e388bfe883c0 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:14:39.216463 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51ca08b-74cf-44f8-b70f-945e51e3d378 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:14:39.217088 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51ca08b-74cf-44f8-b70f-945e51e3d378 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:14:39.217364 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51ca08b-74cf-44f8-b70f-945e51e3d378 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:14:39.217630 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51ca08b-74cf-44f8-b70f-945e51e3d378 None admin] PATH_INFO: `/v3/domains` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:14:39.218250 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51ca08b-74cf-44f8-b70f-945e51e3d378 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:14:39.218546 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51ca08b-74cf-44f8-b70f-945e51e3d378 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:14:39.219893 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51ca08b-74cf-44f8-b70f-945e51e3d378 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:14:39.231448 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 630/1266] 10.222.0.15 () {56 vars in 1109 bytes} [Thu Sep 15 12:14:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:14:39.237202 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51408039-0fab-4178-bc85-6df0dadeb3cc None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:14:39.333247 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51408039-0fab-4178-bc85-6df0dadeb3cc None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:14:39.352568 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:14:39.366215 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba80cfe-4dbe-456e-b3f2-31b74926892d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:14:39.366828 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba80cfe-4dbe-456e-b3f2-31b74926892d None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:14:39.367106 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba80cfe-4dbe-456e-b3f2-31b74926892d None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:14:39.367373 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba80cfe-4dbe-456e-b3f2-31b74926892d None admin] PATH_INFO: `/v3/users/placement` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:14:39.374568 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba80cfe-4dbe-456e-b3f2-31b74926892d None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:14:39.375988 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba80cfe-4dbe-456e-b3f2-31b74926892d None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:14:39.387257 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application [None req-4ba80cfe-4dbe-456e-b3f2-31b74926892d None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 15 12:14:39.387257 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:14:39.387257 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:14:39.387257 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:14:39.387257 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:14:39.387257 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:14:39.387257 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:14:39.387257 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:14:39.387257 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:14:39.387257 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:14:39.387257 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:14:39.387257 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:14:39.387257 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 15 12:14:39.387257 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 15 12:14:39.387257 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 15 12:14:39.387257 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 15 12:14:39.388346 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:14:39.388346 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:14:39.388346 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 15 12:14:39.388346 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 12:14:39.388346 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 15 12:14:39.388346 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 12:14:39.388346 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:14:39.388346 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:14:39.388346 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:14:39.388346 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:14:39.388346 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:14:39.388346 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application with Lock( Sep 15 12:14:39.388346 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:14:39.388346 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:14:39.388346 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:14:39.388346 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:14:39.388346 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:14:39.388346 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:14:39.389384 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:14:39.389384 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:14:39.389384 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 15 12:14:39.389384 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 15 12:14:39.389384 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 15 12:14:39.389384 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 15 12:14:39.389384 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 15 12:14:39.389384 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 15 12:14:39.389384 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 15 12:14:39.389384 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application Sep 15 12:14:39.389384 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 637/1267] 10.222.0.15 () {56 vars in 1108 bytes} [Thu Sep 15 12:14:39 2022] GET /identity/v3/users/placement => generated 87 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:14:39.394057 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51ca08b-74cf-44f8-b70f-945e51e3d378 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:14:39.495526 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51ca08b-74cf-44f8-b70f-945e51e3d378 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:14:39.527490 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238498a6-dfc0-46e5-8030-7e13e6f2ef42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:14:39.528083 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238498a6-dfc0-46e5-8030-7e13e6f2ef42 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:14:39.528358 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238498a6-dfc0-46e5-8030-7e13e6f2ef42 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:14:39.528676 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238498a6-dfc0-46e5-8030-7e13e6f2ef42 None admin] PATH_INFO: `/v3/users` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:14:39.529453 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238498a6-dfc0-46e5-8030-7e13e6f2ef42 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:14:39.529750 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238498a6-dfc0-46e5-8030-7e13e6f2ef42 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:14:39.531150 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-238498a6-dfc0-46e5-8030-7e13e6f2ef42 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:14:39.539437 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:14:39.547919 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 631/1268] 10.222.0.15 () {56 vars in 1143 bytes} [Thu Sep 15 12:14:39 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:14:39.553741 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba80cfe-4dbe-456e-b3f2-31b74926892d None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:14:39.657152 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba80cfe-4dbe-456e-b3f2-31b74926892d None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:14:39.690791 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e43e870-0b4d-4717-92e8-fc2914cb57bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:14:39.691407 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e43e870-0b4d-4717-92e8-fc2914cb57bf None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:14:39.691682 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e43e870-0b4d-4717-92e8-fc2914cb57bf None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:14:39.691946 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e43e870-0b4d-4717-92e8-fc2914cb57bf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:14:39.702580 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e43e870-0b4d-4717-92e8-fc2914cb57bf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:14:39.703990 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e43e870-0b4d-4717-92e8-fc2914cb57bf None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:14:39.717712 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application [None req-7e43e870-0b4d-4717-92e8-fc2914cb57bf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 12:14:39.717712 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:14:39.717712 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:14:39.717712 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:14:39.717712 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:14:39.717712 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:14:39.717712 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:14:39.717712 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:14:39.717712 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:14:39.717712 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:14:39.717712 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:14:39.717712 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:14:39.717712 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 12:14:39.717712 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 12:14:39.717712 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 12:14:39.717712 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 12:14:39.718809 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:14:39.718809 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:14:39.718809 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:14:39.718809 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:14:39.718809 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:14:39.718809 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:14:39.718809 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:14:39.718809 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application with Lock( Sep 15 12:14:39.718809 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:14:39.718809 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:14:39.718809 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:14:39.718809 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:14:39.718809 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:14:39.718809 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:14:39.718809 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:14:39.718809 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:14:39.718809 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 12:14:39.719913 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 12:14:39.719913 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 12:14:39.719913 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application Sep 15 12:14:39.719913 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 638/1269] 10.222.0.15 () {56 vars in 1108 bytes} [Thu Sep 15 12:14:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:14:39.725016 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238498a6-dfc0-46e5-8030-7e13e6f2ef42 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:14:39.821331 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238498a6-dfc0-46e5-8030-7e13e6f2ef42 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:14:39.854069 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8381fc86-99e3-40b3-9d02-eaf679459d43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:14:39.854669 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8381fc86-99e3-40b3-9d02-eaf679459d43 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:14:39.854947 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8381fc86-99e3-40b3-9d02-eaf679459d43 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:14:39.855215 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8381fc86-99e3-40b3-9d02-eaf679459d43 None admin] PATH_INFO: `/v3/domains` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:14:39.855843 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8381fc86-99e3-40b3-9d02-eaf679459d43 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:14:39.856143 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8381fc86-99e3-40b3-9d02-eaf679459d43 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:14:39.857515 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8381fc86-99e3-40b3-9d02-eaf679459d43 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:14:39.868184 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 632/1270] 10.222.0.15 () {56 vars in 1109 bytes} [Thu Sep 15 12:14:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:14:39.874385 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e43e870-0b4d-4717-92e8-fc2914cb57bf None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:14:39.905185 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:14:39.971515 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e43e870-0b4d-4717-92e8-fc2914cb57bf None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:14:40.004957 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a35a780-2539-460c-b14b-eb301ccbce65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:14:40.005578 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a35a780-2539-460c-b14b-eb301ccbce65 None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:14:40.005851 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a35a780-2539-460c-b14b-eb301ccbce65 None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:14:40.006118 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a35a780-2539-460c-b14b-eb301ccbce65 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:14:40.012483 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a35a780-2539-460c-b14b-eb301ccbce65 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:14:40.013934 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a35a780-2539-460c-b14b-eb301ccbce65 None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:14:40.024736 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application [None req-4a35a780-2539-460c-b14b-eb301ccbce65 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 12:14:40.024736 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:14:40.024736 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:14:40.024736 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:14:40.024736 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:14:40.024736 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:14:40.024736 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:14:40.024736 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:14:40.024736 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:14:40.024736 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:14:40.024736 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:14:40.024736 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:14:40.024736 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 15 12:14:40.024736 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 15 12:14:40.024736 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 15 12:14:40.025856 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 15 12:14:40.025856 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:14:40.025856 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:14:40.025856 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:14:40.025856 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:14:40.025856 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:14:40.025856 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:14:40.025856 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:14:40.025856 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application with Lock( Sep 15 12:14:40.025856 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:14:40.025856 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:14:40.025856 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:14:40.025856 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:14:40.025856 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:14:40.025856 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:14:40.025856 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:14:40.025856 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:14:40.026987 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 15 12:14:40.026987 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 15 12:14:40.026987 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 15 12:14:40.026987 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 15 12:14:40.026987 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 15 12:14:40.026987 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 15 12:14:40.026987 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 12:14:40.026987 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application Sep 15 12:14:40.026987 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 639/1271] 10.222.0.15 () {56 vars in 1111 bytes} [Thu Sep 15 12:14:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:14:40.031790 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8381fc86-99e3-40b3-9d02-eaf679459d43 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:14:40.087684 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:14:40.129392 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8381fc86-99e3-40b3-9d02-eaf679459d43 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:14:40.161223 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4efd2c07-c271-4f72-adb9-4f8a5da70dc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:14:40.161822 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efd2c07-c271-4f72-adb9-4f8a5da70dc5 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:14:40.162102 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efd2c07-c271-4f72-adb9-4f8a5da70dc5 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:14:40.162373 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efd2c07-c271-4f72-adb9-4f8a5da70dc5 None admin] PATH_INFO: `/v3/projects` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:14:40.163053 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4efd2c07-c271-4f72-adb9-4f8a5da70dc5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:14:40.163360 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4efd2c07-c271-4f72-adb9-4f8a5da70dc5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:14:40.164784 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4efd2c07-c271-4f72-adb9-4f8a5da70dc5 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:14:40.176663 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 633/1272] 10.222.0.15 () {56 vars in 1148 bytes} [Thu Sep 15 12:14:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:14:40.183196 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a35a780-2539-460c-b14b-eb301ccbce65 None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:14:40.270969 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:14:40.280270 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a35a780-2539-460c-b14b-eb301ccbce65 None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:14:40.312189 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf9ef2a-525b-44b9-b439-dd09f0e6758a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:14:40.312853 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf9ef2a-525b-44b9-b439-dd09f0e6758a None admin] REQUEST_METHOD: `PUT` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:14:40.313135 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf9ef2a-525b-44b9-b439-dd09f0e6758a None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:14:40.313406 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf9ef2a-525b-44b9-b439-dd09f0e6758a None admin] PATH_INFO: `/v3/projects/540a32ef88764e8c8d9201ae22f6ddaf/users/51f64a058bd5440cb5268f228aa6e392/roles/9c769b5c82884766b2c510b1f0f1364c` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:14:40.315252 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf9ef2a-525b-44b9-b439-dd09f0e6758a None admin] RBAC: Authorizing `identity:create_grant(project_id=540a32ef88764e8c8d9201ae22f6ddaf, user_id=51f64a058bd5440cb5268f228aa6e392, role_id=9c769b5c82884766b2c510b1f0f1364c)` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:14:40.316675 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf9ef2a-525b-44b9-b439-dd09f0e6758a None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:14:40.329267 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 640/1273] 10.222.0.15 () {58 vars in 1439 bytes} [Thu Sep 15 12:14:40 2022] PUT /identity/v3/projects/540a32ef88764e8c8d9201ae22f6ddaf/users/51f64a058bd5440cb5268f228aa6e392/roles/9c769b5c82884766b2c510b1f0f1364c => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:14:42.419285 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c3b42e-268e-4bf4-a348-d7af40af4924 None None] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:14:42.419797 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c3b42e-268e-4bf4-a348-d7af40af4924 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:14:42.419797 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c3b42e-268e-4bf4-a348-d7af40af4924 None None] PATH_INFO: `/` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:14:42.421093 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 634/1274] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 12:14:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:14:42.426369 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78aa2c71-675e-47de-aeb9-34985c2d7050 None None] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:14:42.426779 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78aa2c71-675e-47de-aeb9-34985c2d7050 None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:14:42.426919 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78aa2c71-675e-47de-aeb9-34985c2d7050 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:14:42.445821 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.auth.core [None req-78aa2c71-675e-47de-aeb9-34985c2d7050 None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81449) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:14:42.446994 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-78aa2c71-675e-47de-aeb9-34985c2d7050 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:14:42.510968 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 641/1275] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 12:14:42 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 15 12:14:42.777076 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c3b42e-268e-4bf4-a348-d7af40af4924 None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:14:42.872826 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c3b42e-268e-4bf4-a348-d7af40af4924 None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:14:42.889450 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:14:42.905780 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd498235-b13f-432c-8d7f-8d8b41c34f89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:14:42.906412 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd498235-b13f-432c-8d7f-8d8b41c34f89 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:14:42.906707 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd498235-b13f-432c-8d7f-8d8b41c34f89 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:14:42.906975 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd498235-b13f-432c-8d7f-8d8b41c34f89 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:14:42.916591 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd498235-b13f-432c-8d7f-8d8b41c34f89 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Sep 15 12:14:42.921291 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application [None req-bd498235-b13f-432c-8d7f-8d8b41c34f89 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Sep 15 12:14:42.921291 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:14:42.921291 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:14:42.921291 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:14:42.921291 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:14:42.921291 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:14:42.921291 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:14:42.921291 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:14:42.921291 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:14:42.921291 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:14:42.921291 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:14:42.921291 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:14:42.921291 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Sep 15 12:14:42.921291 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_role(role_id) Sep 15 12:14:42.921291 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Sep 15 12:14:42.921291 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Sep 15 12:14:42.922344 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Sep 15 12:14:42.922344 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Sep 15 12:14:42.922344 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Sep 15 12:14:42.922344 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Sep 15 12:14:42.922344 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:14:42.922344 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:14:42.922344 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:14:42.922344 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:14:42.922344 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:14:42.922344 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:14:42.922344 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:14:42.922344 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application with Lock( Sep 15 12:14:42.922344 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:14:42.922344 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:14:42.922344 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:14:42.922344 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:14:42.923795 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:14:42.923795 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:14:42.923795 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:14:42.923795 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:14:42.923795 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Sep 15 12:14:42.923795 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Sep 15 12:14:42.923795 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Sep 15 12:14:42.923795 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Sep 15 12:14:42.923795 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Sep 15 12:14:42.923795 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Sep 15 12:14:42.923795 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Sep 15 12:14:42.923795 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 15 12:14:42.923795 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 635/1276] 10.222.0.15 () {56 vars in 1096 bytes} [Thu Sep 15 12:14:42 2022] GET /identity/v3/roles/admin => generated 83 bytes in 146 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:14:42.928226 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78aa2c71-675e-47de-aeb9-34985c2d7050 None None] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:14:43.007034 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:14:43.026690 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78aa2c71-675e-47de-aeb9-34985c2d7050 None None] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:14:43.059221 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744890cc-6ce0-4c44-845f-6b0714a65316 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:14:43.059768 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744890cc-6ce0-4c44-845f-6b0714a65316 None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:14:43.060049 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744890cc-6ce0-4c44-845f-6b0714a65316 None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:14:43.060320 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744890cc-6ce0-4c44-845f-6b0714a65316 None admin] PATH_INFO: `/v3/roles` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:14:43.060997 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744890cc-6ce0-4c44-845f-6b0714a65316 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81449) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:14:43.061300 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744890cc-6ce0-4c44-845f-6b0714a65316 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:14:43.062657 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744890cc-6ce0-4c44-845f-6b0714a65316 None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:14:43.070444 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 642/1277] 10.222.0.15 () {56 vars in 1099 bytes} [Thu Sep 15 12:14:42 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:14:43.076712 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd498235-b13f-432c-8d7f-8d8b41c34f89 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:14:43.171814 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd498235-b13f-432c-8d7f-8d8b41c34f89 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:14:43.203288 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244729b3-a409-4e7a-858a-a455435d87cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:14:43.203902 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244729b3-a409-4e7a-858a-a455435d87cf None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:14:43.204175 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244729b3-a409-4e7a-858a-a455435d87cf None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:14:43.204440 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244729b3-a409-4e7a-858a-a455435d87cf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:14:43.214889 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244729b3-a409-4e7a-858a-a455435d87cf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:14:43.216302 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244729b3-a409-4e7a-858a-a455435d87cf None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:14:43.229909 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application [None req-244729b3-a409-4e7a-858a-a455435d87cf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 12:14:43.229909 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:14:43.229909 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:14:43.229909 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:14:43.229909 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:14:43.229909 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:14:43.229909 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:14:43.229909 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:14:43.229909 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:14:43.229909 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:14:43.229909 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:14:43.229909 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:14:43.229909 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 12:14:43.229909 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 12:14:43.229909 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 12:14:43.229909 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 12:14:43.231094 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:14:43.231094 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:14:43.231094 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:14:43.231094 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:14:43.231094 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:14:43.231094 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:14:43.231094 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:14:43.231094 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application with Lock( Sep 15 12:14:43.231094 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:14:43.231094 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:14:43.231094 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:14:43.231094 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:14:43.231094 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:14:43.231094 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:14:43.231094 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:14:43.231094 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:14:43.231094 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 12:14:43.232032 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 12:14:43.232032 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 12:14:43.232032 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 15 12:14:43.232032 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 636/1278] 10.222.0.15 () {56 vars in 1108 bytes} [Thu Sep 15 12:14:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:14:43.236434 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744890cc-6ce0-4c44-845f-6b0714a65316 None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:14:43.335768 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744890cc-6ce0-4c44-845f-6b0714a65316 None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:14:43.367769 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4466d8-2e89-4bdf-b741-03fead5a5ec8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:14:43.368904 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4466d8-2e89-4bdf-b741-03fead5a5ec8 None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:14:43.369187 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4466d8-2e89-4bdf-b741-03fead5a5ec8 None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:14:43.369455 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4466d8-2e89-4bdf-b741-03fead5a5ec8 None admin] PATH_INFO: `/v3/domains` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:14:43.370083 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4466d8-2e89-4bdf-b741-03fead5a5ec8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81449) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:14:43.370381 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4466d8-2e89-4bdf-b741-03fead5a5ec8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:14:43.371726 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4466d8-2e89-4bdf-b741-03fead5a5ec8 None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:14:43.383008 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 643/1279] 10.222.0.15 () {56 vars in 1109 bytes} [Thu Sep 15 12:14:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:14:43.389171 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244729b3-a409-4e7a-858a-a455435d87cf None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:14:43.418259 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:14:43.483939 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244729b3-a409-4e7a-858a-a455435d87cf None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:14:43.516020 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0265df2-efec-4d1a-8b26-13cfd2769353 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:14:43.516626 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0265df2-efec-4d1a-8b26-13cfd2769353 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:14:43.516928 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0265df2-efec-4d1a-8b26-13cfd2769353 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:14:43.517196 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0265df2-efec-4d1a-8b26-13cfd2769353 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:14:43.524141 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0265df2-efec-4d1a-8b26-13cfd2769353 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:14:43.525592 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0265df2-efec-4d1a-8b26-13cfd2769353 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:14:43.536152 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application [None req-d0265df2-efec-4d1a-8b26-13cfd2769353 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Sep 15 12:14:43.536152 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:14:43.536152 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:14:43.536152 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:14:43.536152 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:14:43.536152 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:14:43.536152 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:14:43.536152 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:14:43.536152 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:14:43.536152 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:14:43.536152 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:14:43.536152 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:14:43.536152 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Sep 15 12:14:43.536152 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_user(user_id) Sep 15 12:14:43.536152 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Sep 15 12:14:43.536152 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Sep 15 12:14:43.537278 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:14:43.537278 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:14:43.537278 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Sep 15 12:14:43.537278 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 12:14:43.537278 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Sep 15 12:14:43.537278 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Sep 15 12:14:43.537278 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:14:43.537278 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:14:43.537278 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:14:43.537278 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:14:43.537278 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:14:43.537278 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application with Lock( Sep 15 12:14:43.537278 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:14:43.537278 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:14:43.537278 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:14:43.537278 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:14:43.537278 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:14:43.537278 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:14:43.538490 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:14:43.538490 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:14:43.538490 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Sep 15 12:14:43.538490 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Sep 15 12:14:43.538490 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Sep 15 12:14:43.538490 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Sep 15 12:14:43.538490 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Sep 15 12:14:43.538490 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Sep 15 12:14:43.538490 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Sep 15 12:14:43.538490 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 15 12:14:43.538490 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 637/1280] 10.222.0.15 () {56 vars in 1108 bytes} [Thu Sep 15 12:14:43 2022] GET /identity/v3/users/placement => generated 87 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:14:43.542736 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4466d8-2e89-4bdf-b741-03fead5a5ec8 None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:14:43.546426 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:14:43.641339 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4466d8-2e89-4bdf-b741-03fead5a5ec8 None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:14:43.673904 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87df8a86-0453-4866-a3a0-086a680c55b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:14:43.674497 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87df8a86-0453-4866-a3a0-086a680c55b5 None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:14:43.674776 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87df8a86-0453-4866-a3a0-086a680c55b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:14:43.675046 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87df8a86-0453-4866-a3a0-086a680c55b5 None admin] PATH_INFO: `/v3/users` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:14:43.675758 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87df8a86-0453-4866-a3a0-086a680c55b5 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81449) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:14:43.676061 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87df8a86-0453-4866-a3a0-086a680c55b5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:14:43.677485 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87df8a86-0453-4866-a3a0-086a680c55b5 None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:14:43.694331 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 644/1281] 10.222.0.15 () {56 vars in 1143 bytes} [Thu Sep 15 12:14:43 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:14:43.700083 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0265df2-efec-4d1a-8b26-13cfd2769353 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:14:43.785733 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:14:43.795682 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0265df2-efec-4d1a-8b26-13cfd2769353 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:14:43.827728 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb59138-dea8-4e10-a1fb-0152c87ce85e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:14:43.828312 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb59138-dea8-4e10-a1fb-0152c87ce85e None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:14:43.828585 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb59138-dea8-4e10-a1fb-0152c87ce85e None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:14:43.828877 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb59138-dea8-4e10-a1fb-0152c87ce85e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:14:43.892203 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb59138-dea8-4e10-a1fb-0152c87ce85e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:14:43.893583 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb59138-dea8-4e10-a1fb-0152c87ce85e None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:14:43.978204 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:14:43.988749 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 638/1282] 10.222.0.15 () {58 vars in 1282 bytes} [Thu Sep 15 12:14:43 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 289 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Sep 15 12:14:43.995060 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87df8a86-0453-4866-a3a0-086a680c55b5 None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:14:44.055049 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:14:44.092683 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87df8a86-0453-4866-a3a0-086a680c55b5 None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:14:44.125860 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6e6347-6b25-4be9-90ba-6ed09e3766ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:14:44.126482 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6e6347-6b25-4be9-90ba-6ed09e3766ef None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:14:44.126762 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6e6347-6b25-4be9-90ba-6ed09e3766ef None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:14:44.127034 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6e6347-6b25-4be9-90ba-6ed09e3766ef None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:14:44.136677 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae6e6347-6b25-4be9-90ba-6ed09e3766ef None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:14:44.138135 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae6e6347-6b25-4be9-90ba-6ed09e3766ef None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:14:44.153713 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application [None req-ae6e6347-6b25-4be9-90ba-6ed09e3766ef None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 12:14:44.153713 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:14:44.153713 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:14:44.153713 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:14:44.153713 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:14:44.153713 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:14:44.153713 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:14:44.153713 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:14:44.153713 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:14:44.153713 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:14:44.153713 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:14:44.153713 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:14:44.153713 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Sep 15 12:14:44.153713 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Sep 15 12:14:44.153713 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Sep 15 12:14:44.153713 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Sep 15 12:14:44.154777 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:14:44.154777 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:14:44.154777 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:14:44.154777 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:14:44.154777 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:14:44.154777 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:14:44.154777 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:14:44.154777 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application with Lock( Sep 15 12:14:44.154777 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:14:44.154777 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:14:44.154777 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:14:44.154777 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:14:44.154777 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:14:44.154777 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:14:44.154777 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:14:44.154777 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:14:44.154777 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Sep 15 12:14:44.155857 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Sep 15 12:14:44.155857 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Sep 15 12:14:44.155857 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application Sep 15 12:14:44.155857 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 645/1283] 10.222.0.15 () {56 vars in 1108 bytes} [Thu Sep 15 12:14:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:14:44.160408 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb59138-dea8-4e10-a1fb-0152c87ce85e None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:14:44.254109 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb59138-dea8-4e10-a1fb-0152c87ce85e None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:14:44.286562 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13574e94-f82a-4bc4-a60c-0d52c96ff903 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:14:44.287149 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13574e94-f82a-4bc4-a60c-0d52c96ff903 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:14:44.287422 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13574e94-f82a-4bc4-a60c-0d52c96ff903 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:14:44.287686 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13574e94-f82a-4bc4-a60c-0d52c96ff903 None admin] PATH_INFO: `/v3/domains` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:14:44.288308 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13574e94-f82a-4bc4-a60c-0d52c96ff903 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:14:44.288601 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13574e94-f82a-4bc4-a60c-0d52c96ff903 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:14:44.289970 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13574e94-f82a-4bc4-a60c-0d52c96ff903 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:14:44.301038 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 639/1284] 10.222.0.15 () {56 vars in 1109 bytes} [Thu Sep 15 12:14:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:14:44.306446 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6e6347-6b25-4be9-90ba-6ed09e3766ef None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:14:44.401686 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6e6347-6b25-4be9-90ba-6ed09e3766ef None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:14:44.414448 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:14:44.433659 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a679233-d13a-41b9-acd3-541605bae0df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:14:44.434265 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a679233-d13a-41b9-acd3-541605bae0df None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:14:44.434538 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a679233-d13a-41b9-acd3-541605bae0df None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:14:44.434804 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a679233-d13a-41b9-acd3-541605bae0df None admin] PATH_INFO: `/v3/projects/service` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:14:44.441090 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a679233-d13a-41b9-acd3-541605bae0df None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:14:44.442501 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a679233-d13a-41b9-acd3-541605bae0df None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:14:44.453791 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application [None req-2a679233-d13a-41b9-acd3-541605bae0df None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 12:14:44.453791 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:14:44.453791 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:14:44.453791 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:14:44.453791 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:14:44.453791 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:14:44.453791 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:14:44.453791 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:14:44.453791 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:14:44.453791 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:14:44.453791 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:14:44.453791 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:14:44.453791 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Sep 15 12:14:44.453791 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self._get_project(project_id) Sep 15 12:14:44.453791 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Sep 15 12:14:44.454819 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Sep 15 12:14:44.454819 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:14:44.454819 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:14:44.454819 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:14:44.454819 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:14:44.454819 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:14:44.454819 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:14:44.454819 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:14:44.454819 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application with Lock( Sep 15 12:14:44.454819 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:14:44.454819 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:14:44.454819 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:14:44.454819 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:14:44.454819 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:14:44.454819 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:14:44.454819 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:14:44.454819 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:14:44.456090 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Sep 15 12:14:44.456090 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Sep 15 12:14:44.456090 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Sep 15 12:14:44.456090 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Sep 15 12:14:44.456090 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Sep 15 12:14:44.456090 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Sep 15 12:14:44.456090 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Sep 15 12:14:44.456090 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: ERROR keystone.server.flask.application Sep 15 12:14:44.456090 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 646/1285] 10.222.0.15 () {56 vars in 1111 bytes} [Thu Sep 15 12:14:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:14:44.460415 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13574e94-f82a-4bc4-a60c-0d52c96ff903 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:14:44.508702 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:14:44.557397 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13574e94-f82a-4bc4-a60c-0d52c96ff903 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:14:44.589827 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb4fdac-473c-4bab-82ba-cc7a19a4bd96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:14:44.590421 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb4fdac-473c-4bab-82ba-cc7a19a4bd96 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:14:44.590702 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb4fdac-473c-4bab-82ba-cc7a19a4bd96 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:14:44.590971 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb4fdac-473c-4bab-82ba-cc7a19a4bd96 None admin] PATH_INFO: `/v3/projects` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:14:44.591645 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb4fdac-473c-4bab-82ba-cc7a19a4bd96 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:14:44.591950 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb4fdac-473c-4bab-82ba-cc7a19a4bd96 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:14:44.593389 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb4fdac-473c-4bab-82ba-cc7a19a4bd96 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:14:44.606387 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 640/1286] 10.222.0.15 () {56 vars in 1148 bytes} [Thu Sep 15 12:14:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:14:44.612284 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a679233-d13a-41b9-acd3-541605bae0df None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:14:44.708267 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a679233-d13a-41b9-acd3-541605bae0df None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:14:44.741139 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83f7448-1fe2-4293-8ee6-04a9c060287a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:14:44.741729 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83f7448-1fe2-4293-8ee6-04a9c060287a None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:14:44.742008 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83f7448-1fe2-4293-8ee6-04a9c060287a None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:14:44.742286 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83f7448-1fe2-4293-8ee6-04a9c060287a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:14:44.742994 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d83f7448-1fe2-4293-8ee6-04a9c060287a None admin] RBAC: Adding query filter params (role.id=9c769b5c82884766b2c510b1f0f1364c, scope.project.id=540a32ef88764e8c8d9201ae22f6ddaf, user.id=51f64a058bd5440cb5268f228aa6e392) {{(pid=81449) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:14:44.745223 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d83f7448-1fe2-4293-8ee6-04a9c060287a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:14:44.745360 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d83f7448-1fe2-4293-8ee6-04a9c060287a None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:14:44.752457 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 647/1287] 10.222.0.15 () {56 vars in 1374 bytes} [Thu Sep 15 12:14:44 2022] GET /identity/v3/role_assignments?user.id=51f64a058bd5440cb5268f228aa6e392&scope.project.id=540a32ef88764e8c8d9201ae22f6ddaf&role.id=9c769b5c82884766b2c510b1f0f1364c => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:14:46.863009 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a464c0d-52a5-4375-ab1c-9ab599efeb73 None None] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:14:46.863550 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a464c0d-52a5-4375-ab1c-9ab599efeb73 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:14:46.863550 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a464c0d-52a5-4375-ab1c-9ab599efeb73 None None] PATH_INFO: `/` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:14:46.864888 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 641/1288] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 12:14:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:14:46.869942 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bde02a-0b4e-413a-b4fc-2493c73c8eed None None] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:14:46.870313 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bde02a-0b4e-413a-b4fc-2493c73c8eed None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:14:46.870489 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bde02a-0b4e-413a-b4fc-2493c73c8eed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:14:46.888540 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.auth.core [None req-99bde02a-0b4e-413a-b4fc-2493c73c8eed None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81449) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:14:46.889737 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-99bde02a-0b4e-413a-b4fc-2493c73c8eed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:14:46.918163 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:14:46.953622 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 648/1289] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 12:14:46 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 15 12:14:47.219738 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a464c0d-52a5-4375-ab1c-9ab599efeb73 None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:14:47.316614 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a464c0d-52a5-4375-ab1c-9ab599efeb73 None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:14:47.327296 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:14:47.349971 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab73fd5-87a6-4938-9271-02e11f874368 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:14:47.350593 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab73fd5-87a6-4938-9271-02e11f874368 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:14:47.350871 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab73fd5-87a6-4938-9271-02e11f874368 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:14:47.351141 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab73fd5-87a6-4938-9271-02e11f874368 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:14:47.351712 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab73fd5-87a6-4938-9271-02e11f874368 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:14:47.353110 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab73fd5-87a6-4938-9271-02e11f874368 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:14:47.361542 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application [None req-eab73fd5-87a6-4938-9271-02e11f874368 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 15 12:14:47.361542 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:14:47.361542 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 15 12:14:47.361542 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 15 12:14:47.361542 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 15 12:14:47.361542 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 15 12:14:47.361542 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 15 12:14:47.361542 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 15 12:14:47.361542 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 15 12:14:47.361542 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 15 12:14:47.361542 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 15 12:14:47.361542 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 15 12:14:47.361542 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:14:47.361542 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:14:47.361542 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:14:47.361542 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:14:47.361542 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:14:47.362702 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:14:47.362702 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:14:47.362702 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:14:47.362702 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:14:47.362702 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:14:47.362702 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:14:47.362702 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 15 12:14:47.362702 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 15 12:14:47.362702 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 15 12:14:47.362702 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 15 12:14:47.362702 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:14:47.362702 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:14:47.362702 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:14:47.362702 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:14:47.362702 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:14:47.362702 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:14:47.363598 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:14:47.363598 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application with Lock( Sep 15 12:14:47.363598 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:14:47.363598 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:14:47.363598 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:14:47.363598 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:14:47.363598 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:14:47.363598 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:14:47.363598 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:14:47.363598 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:14:47.363598 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 15 12:14:47.363598 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 15 12:14:47.363598 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 15 12:14:47.363598 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 15 12:14:47.363598 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 642/1290] 10.222.0.15 () {56 vars in 1117 bytes} [Thu Sep 15 12:14:47 2022] GET /identity/v3/services/placement => generated 90 bytes in 143 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:14:47.369024 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99bde02a-0b4e-413a-b4fc-2493c73c8eed None None] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:14:47.462197 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99bde02a-0b4e-413a-b4fc-2493c73c8eed None None] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:14:47.495016 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e007c6-ff91-4f50-be81-4acef2c7c4c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:14:47.495545 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e007c6-ff91-4f50-be81-4acef2c7c4c3 None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:14:47.495819 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e007c6-ff91-4f50-be81-4acef2c7c4c3 None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:14:47.496087 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e007c6-ff91-4f50-be81-4acef2c7c4c3 None admin] PATH_INFO: `/v3/services` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:14:47.496692 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e007c6-ff91-4f50-be81-4acef2c7c4c3 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81449) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:14:47.497013 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e007c6-ff91-4f50-be81-4acef2c7c4c3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:14:47.498353 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e007c6-ff91-4f50-be81-4acef2c7c4c3 None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:14:47.502996 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 649/1291] 10.222.0.15 () {56 vars in 1116 bytes} [Thu Sep 15 12:14:47 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:14:47.508644 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab73fd5-87a6-4938-9271-02e11f874368 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:14:47.604212 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab73fd5-87a6-4938-9271-02e11f874368 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:14:47.641295 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78013159-34e3-492b-b31e-9dd5e8c19024 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:14:47.641900 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78013159-34e3-492b-b31e-9dd5e8c19024 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:14:47.642183 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78013159-34e3-492b-b31e-9dd5e8c19024 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:14:47.642457 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78013159-34e3-492b-b31e-9dd5e8c19024 None admin] PATH_INFO: `/v3/services` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:14:47.643101 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78013159-34e3-492b-b31e-9dd5e8c19024 None admin] RBAC: Adding query filter params (type=placement) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:14:47.643379 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78013159-34e3-492b-b31e-9dd5e8c19024 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:14:47.644727 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78013159-34e3-492b-b31e-9dd5e8c19024 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:14:47.649387 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 643/1292] 10.222.0.15 () {56 vars in 1116 bytes} [Thu Sep 15 12:14:47 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:14:49.766879 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46aacde3-7d14-4db3-8a30-1de10f353a87 None None] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:14:49.767476 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46aacde3-7d14-4db3-8a30-1de10f353a87 None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:14:49.767476 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46aacde3-7d14-4db3-8a30-1de10f353a87 None None] PATH_INFO: `/` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:14:49.768779 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 650/1293] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 12:14:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:14:49.773948 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109d61a9-b5d6-41aa-943d-312698e1d45b None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:14:49.774350 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109d61a9-b5d6-41aa-943d-312698e1d45b None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:14:49.775067 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109d61a9-b5d6-41aa-943d-312698e1d45b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:14:49.792646 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-109d61a9-b5d6-41aa-943d-312698e1d45b None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:14:49.793857 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-109d61a9-b5d6-41aa-943d-312698e1d45b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:14:49.822142 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:14:49.858620 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 644/1294] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 12:14:49 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 15 12:14:50.131052 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46aacde3-7d14-4db3-8a30-1de10f353a87 None None] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:14:50.161668 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:14:50.229043 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46aacde3-7d14-4db3-8a30-1de10f353a87 None None] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:14:50.262063 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5fbdeb-03d7-4268-b05e-294cbf15f64c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:14:50.262652 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5fbdeb-03d7-4268-b05e-294cbf15f64c None admin] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:14:50.262927 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5fbdeb-03d7-4268-b05e-294cbf15f64c None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:14:50.263194 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5fbdeb-03d7-4268-b05e-294cbf15f64c None admin] PATH_INFO: `/v3/services` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:14:50.263834 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5fbdeb-03d7-4268-b05e-294cbf15f64c None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:14:50.265211 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5fbdeb-03d7-4268-b05e-294cbf15f64c None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:14:50.279592 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 651/1295] 10.222.0.15 () {60 vars in 1141 bytes} [Thu Sep 15 12:14:50 2022] POST /identity/v3/services => generated 246 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 12:14:52.428334 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caaa9320-fdaa-48cc-89c7-adb56ad1abcd None None] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:14:52.428910 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caaa9320-fdaa-48cc-89c7-adb56ad1abcd None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:14:52.428910 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caaa9320-fdaa-48cc-89c7-adb56ad1abcd None None] PATH_INFO: `/` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:14:52.430321 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 645/1296] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 12:14:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:14:52.435446 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7a9c4d-8338-413d-a75f-236f4900ad40 None None] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:14:52.435868 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7a9c4d-8338-413d-a75f-236f4900ad40 None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:14:52.435972 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7a9c4d-8338-413d-a75f-236f4900ad40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:14:52.454100 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.auth.core [None req-8e7a9c4d-8338-413d-a75f-236f4900ad40 None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81449) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:14:52.455264 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-8e7a9c4d-8338-413d-a75f-236f4900ad40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:14:52.525429 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 652/1297] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 12:14:52 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 15 12:14:52.791494 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caaa9320-fdaa-48cc-89c7-adb56ad1abcd None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:14:52.889068 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caaa9320-fdaa-48cc-89c7-adb56ad1abcd None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:14:52.921195 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16721c5-b212-4588-ba64-f92f3242d019 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:14:52.921811 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16721c5-b212-4588-ba64-f92f3242d019 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:14:52.922088 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16721c5-b212-4588-ba64-f92f3242d019 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:14:52.922358 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16721c5-b212-4588-ba64-f92f3242d019 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:14:52.922929 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16721c5-b212-4588-ba64-f92f3242d019 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:14:52.924276 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16721c5-b212-4588-ba64-f92f3242d019 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:14:52.933359 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application [None req-f16721c5-b212-4588-ba64-f92f3242d019 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 15 12:14:52.933359 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:14:52.933359 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 15 12:14:52.933359 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 15 12:14:52.933359 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 15 12:14:52.933359 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 15 12:14:52.933359 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 15 12:14:52.933359 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 15 12:14:52.933359 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 15 12:14:52.933359 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 15 12:14:52.933359 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 15 12:14:52.933359 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 15 12:14:52.933359 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:14:52.933359 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:14:52.933359 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:14:52.933359 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:14:52.933359 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:14:52.934658 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:14:52.934658 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:14:52.934658 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:14:52.934658 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:14:52.934658 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:14:52.934658 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:14:52.934658 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 15 12:14:52.934658 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 15 12:14:52.934658 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 15 12:14:52.934658 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 15 12:14:52.934658 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:14:52.934658 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:14:52.934658 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:14:52.934658 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:14:52.934658 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:14:52.934658 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:14:52.935841 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:14:52.935841 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application with Lock( Sep 15 12:14:52.935841 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:14:52.935841 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:14:52.935841 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:14:52.935841 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:14:52.935841 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:14:52.935841 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:14:52.935841 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:14:52.935841 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:14:52.935841 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 15 12:14:52.935841 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 15 12:14:52.935841 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 15 12:14:52.935841 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 15 12:14:52.935841 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 646/1298] 10.222.0.15 () {56 vars in 1117 bytes} [Thu Sep 15 12:14:52 2022] GET /identity/v3/services/placement => generated 90 bytes in 143 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:14:52.940309 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7a9c4d-8338-413d-a75f-236f4900ad40 None None] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:14:52.958012 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:14:53.037156 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7a9c4d-8338-413d-a75f-236f4900ad40 None None] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:14:53.069512 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f88e8da-a2d6-4af3-bdf6-c85aeadfa7be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:14:53.070051 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f88e8da-a2d6-4af3-bdf6-c85aeadfa7be None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:14:53.070327 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f88e8da-a2d6-4af3-bdf6-c85aeadfa7be None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:14:53.070597 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f88e8da-a2d6-4af3-bdf6-c85aeadfa7be None admin] PATH_INFO: `/v3/services` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:14:53.071237 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f88e8da-a2d6-4af3-bdf6-c85aeadfa7be None admin] RBAC: Adding query filter params (name=placement) {{(pid=81449) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:14:53.071538 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f88e8da-a2d6-4af3-bdf6-c85aeadfa7be None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:14:53.072924 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f88e8da-a2d6-4af3-bdf6-c85aeadfa7be None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:14:53.077828 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 653/1299] 10.222.0.15 () {56 vars in 1116 bytes} [Thu Sep 15 12:14:52 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:14:53.083857 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16721c5-b212-4588-ba64-f92f3242d019 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:14:53.116154 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:14:53.182351 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16721c5-b212-4588-ba64-f92f3242d019 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:14:53.214150 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286aaaa8-261d-426e-8d10-7fe43fdb6e16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:14:53.214745 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286aaaa8-261d-426e-8d10-7fe43fdb6e16 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:14:53.215024 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286aaaa8-261d-426e-8d10-7fe43fdb6e16 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:14:53.215294 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286aaaa8-261d-426e-8d10-7fe43fdb6e16 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:14:53.215980 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286aaaa8-261d-426e-8d10-7fe43fdb6e16 None admin] RBAC: Adding query filter params (interface=public, service_id=18c597ce92fc46eea55c28dc034f3af1, region_id=RegionOne) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:14:53.216286 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286aaaa8-261d-426e-8d10-7fe43fdb6e16 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:14:53.217652 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286aaaa8-261d-426e-8d10-7fe43fdb6e16 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:14:53.222427 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 647/1300] 10.222.0.15 () {56 vars in 1251 bytes} [Thu Sep 15 12:14:53 2022] GET /identity/v3/endpoints?service_id=18c597ce92fc46eea55c28dc034f3af1&interface=public®ion_id=RegionOne => generated 194 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:14:53.228747 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f88e8da-a2d6-4af3-bdf6-c85aeadfa7be None admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:14:53.306954 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:14:53.324926 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f88e8da-a2d6-4af3-bdf6-c85aeadfa7be None admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:14:53.358096 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661cf977-d2d7-485f-8d3d-61d6d2c37042 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:14:53.358688 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661cf977-d2d7-485f-8d3d-61d6d2c37042 None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:14:53.358968 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661cf977-d2d7-485f-8d3d-61d6d2c37042 None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:14:53.359238 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661cf977-d2d7-485f-8d3d-61d6d2c37042 None admin] PATH_INFO: `/v3/services` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:14:53.359829 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661cf977-d2d7-485f-8d3d-61d6d2c37042 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:14:53.361219 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661cf977-d2d7-485f-8d3d-61d6d2c37042 None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:14:53.368177 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 654/1301] 10.222.0.15 () {56 vars in 1087 bytes} [Thu Sep 15 12:14:53 2022] GET /identity/v3/services => generated 2191 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 12:14:55.484299 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb202653-9102-47c0-9d3c-a788ea62eb89 None None] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:14:55.484869 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb202653-9102-47c0-9d3c-a788ea62eb89 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:14:55.484869 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb202653-9102-47c0-9d3c-a788ea62eb89 None None] PATH_INFO: `/` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:14:55.486192 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 648/1302] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 12:14:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:14:55.491410 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c8c744-bd77-4733-94b9-b47400730945 None None] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:14:55.491829 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c8c744-bd77-4733-94b9-b47400730945 None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:14:55.491942 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c8c744-bd77-4733-94b9-b47400730945 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:14:55.509685 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.auth.core [None req-a4c8c744-bd77-4733-94b9-b47400730945 None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81449) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:14:55.510854 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-a4c8c744-bd77-4733-94b9-b47400730945 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:14:55.574164 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 655/1303] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 12:14:55 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Sep 15 12:14:55.849985 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb202653-9102-47c0-9d3c-a788ea62eb89 None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:14:55.941219 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:14:55.948857 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb202653-9102-47c0-9d3c-a788ea62eb89 None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:14:55.982160 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70604009-4463-4787-90b6-e5ab2573f434 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:14:55.982802 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70604009-4463-4787-90b6-e5ab2573f434 None admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:14:55.983080 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70604009-4463-4787-90b6-e5ab2573f434 None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:14:55.983348 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70604009-4463-4787-90b6-e5ab2573f434 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:14:55.983927 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70604009-4463-4787-90b6-e5ab2573f434 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:14:55.985314 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70604009-4463-4787-90b6-e5ab2573f434 None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:14:55.994396 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application [None req-70604009-4463-4787-90b6-e5ab2573f434 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Sep 15 12:14:55.994396 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:14:55.994396 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Sep 15 12:14:55.994396 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Sep 15 12:14:55.994396 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Sep 15 12:14:55.994396 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Sep 15 12:14:55.994396 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Sep 15 12:14:55.994396 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 15 12:14:55.994396 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 15 12:14:55.994396 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 15 12:14:55.994396 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Sep 15 12:14:55.994396 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 15 12:14:55.994396 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Traceback (most recent call last): Sep 15 12:14:55.994396 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Sep 15 12:14:55.994396 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application rv = self.dispatch_request() Sep 15 12:14:55.994396 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Sep 15 12:14:55.994396 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Sep 15 12:14:55.996930 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Sep 15 12:14:55.996930 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Sep 15 12:14:55.996930 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Sep 15 12:14:55.996930 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Sep 15 12:14:55.996930 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Sep 15 12:14:55.996930 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Sep 15 12:14:55.996930 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Sep 15 12:14:55.996930 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._get_service(service_id) Sep 15 12:14:55.996930 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Sep 15 12:14:55.996930 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Sep 15 12:14:55.996930 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Sep 15 12:14:55.996930 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Sep 15 12:14:55.996930 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Sep 15 12:14:55.996930 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Sep 15 12:14:55.996930 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Sep 15 12:14:55.996930 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.get_or_create( Sep 15 12:14:55.998041 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Sep 15 12:14:55.998041 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application with Lock( Sep 15 12:14:55.998041 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Sep 15 12:14:55.998041 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self._enter() Sep 15 12:14:55.998041 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Sep 15 12:14:55.998041 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Sep 15 12:14:55.998041 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Sep 15 12:14:55.998041 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application return self.creator() Sep 15 12:14:55.998041 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Sep 15 12:14:55.998041 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application created_value = creator( Sep 15 12:14:55.998041 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Sep 15 12:14:55.998041 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Sep 15 12:14:55.998041 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Sep 15 12:14:55.998041 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: ERROR keystone.server.flask.application Sep 15 12:14:55.998041 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 649/1304] 10.222.0.15 () {56 vars in 1117 bytes} [Thu Sep 15 12:14:55 2022] GET /identity/v3/services/placement => generated 90 bytes in 146 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Sep 15 12:14:56.001428 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c8c744-bd77-4733-94b9-b47400730945 None None] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:14:56.082964 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:14:56.105886 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c8c744-bd77-4733-94b9-b47400730945 None None] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:14:56.139818 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd550b1-1e33-440f-9ab7-0a4a3da60018 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:14:56.140431 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd550b1-1e33-440f-9ab7-0a4a3da60018 None admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:14:56.140711 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd550b1-1e33-440f-9ab7-0a4a3da60018 None admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:14:56.141018 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd550b1-1e33-440f-9ab7-0a4a3da60018 None admin] PATH_INFO: `/v3/services` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:14:56.141638 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd550b1-1e33-440f-9ab7-0a4a3da60018 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81449) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:14:56.141942 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd550b1-1e33-440f-9ab7-0a4a3da60018 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:14:56.143330 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd550b1-1e33-440f-9ab7-0a4a3da60018 None admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:14:56.148304 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 656/1305] 10.222.0.15 () {56 vars in 1116 bytes} [Thu Sep 15 12:14:56 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:14:56.154079 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70604009-4463-4787-90b6-e5ab2573f434 None admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:14:56.249627 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70604009-4463-4787-90b6-e5ab2573f434 None admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:14:56.281666 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66153d0-c73c-4662-892b-e188fd23114a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:14:56.282260 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66153d0-c73c-4662-892b-e188fd23114a None admin] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:14:56.282538 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66153d0-c73c-4662-892b-e188fd23114a None admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:14:56.282809 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66153d0-c73c-4662-892b-e188fd23114a None admin] PATH_INFO: `/v3/endpoints` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:14:56.283470 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66153d0-c73c-4662-892b-e188fd23114a None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:14:56.284829 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66153d0-c73c-4662-892b-e188fd23114a None admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:14:56.303183 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 650/1306] 10.222.0.15 () {60 vars in 1144 bytes} [Thu Sep 15 12:14:56 2022] POST /identity/v3/endpoints => generated 331 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 12:15:06.145528 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=30 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:15:06.299153 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:16:00.046072 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e50ab17-5e58-4591-8bd0-b655271ea88e None None] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:16:00.046545 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e50ab17-5e58-4591-8bd0-b655271ea88e None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:16:00.046810 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e50ab17-5e58-4591-8bd0-b655271ea88e None None] PATH_INFO: `/` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:16:00.053082 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1adbd0-97db-4f97-8962-c2be0baa0b79 None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:16:00.053500 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1adbd0-97db-4f97-8962-c2be0baa0b79 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:16:00.053805 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1adbd0-97db-4f97-8962-c2be0baa0b79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:16:00.058302 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 657/1307] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 12:16:00 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:16:00.200602 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-ea1adbd0-97db-4f97-8962-c2be0baa0b79 None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:16:00.202407 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-ea1adbd0-97db-4f97-8962-c2be0baa0b79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:16:00.324523 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 651/1308] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 12:16:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 272 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 12:16:00.618565 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aefb2ce-340e-4416-9f8b-19e99a3f5f22 None None] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:16:00.619072 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aefb2ce-340e-4416-9f8b-19e99a3f5f22 None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:16:00.619463 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aefb2ce-340e-4416-9f8b-19e99a3f5f22 None None] PATH_INFO: `/` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:16:00.620973 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 658/1309] 10.222.0.15 () {54 vars in 965 bytes} [Thu Sep 15 12:16:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:16:00.627483 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbdc0e3-d7a0-4f6b-acb1-4f6b107d71ea None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:16:00.627912 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbdc0e3-d7a0-4f6b-acb1-4f6b107d71ea None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:16:00.628293 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbdc0e3-d7a0-4f6b-acb1-4f6b107d71ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:16:00.718293 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-dcbdc0e3-d7a0-4f6b-acb1-4f6b107d71ea None None] MFA Rules not processed for user `bfbb4d6809064f9cb9e06626ed7d6814`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:16:00.719639 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-dcbdc0e3-d7a0-4f6b-acb1-4f6b107d71ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:16:00.742209 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:16:00.802726 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 652/1310] 10.222.0.15 () {58 vars in 1062 bytes} [Thu Sep 15 12:16:00 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:16:00.811922 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aefb2ce-340e-4416-9f8b-19e99a3f5f22 None None] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:16:00.813695 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-2aefb2ce-340e-4416-9f8b-19e99a3f5f22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:16:00.925104 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aefb2ce-340e-4416-9f8b-19e99a3f5f22 None None] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:16:01.005808 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c275d1-5f9c-41f7-a561-fe40a9dd3378 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfbb4d6809064f9cb9e06626ed7d6814', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '540a32ef88764e8c8d9201ae22f6ddaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:16:01.006416 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c275d1-5f9c-41f7-a561-fe40a9dd3378 service neutron] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:16:01.006691 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c275d1-5f9c-41f7-a561-fe40a9dd3378 service neutron] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:16:01.006958 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c275d1-5f9c-41f7-a561-fe40a9dd3378 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:16:01.008532 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-c8c275d1-5f9c-41f7-a561-fe40a9dd3378 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:16:01.064812 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:16:01.122862 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c275d1-5f9c-41f7-a561-fe40a9dd3378 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:16:01.124290 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c275d1-5f9c-41f7-a561-fe40a9dd3378 service neutron] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:16:01.143186 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 659/1311] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 12:16:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 12:16:03.607483 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5891833c-e0d8-4a89-a3a8-1a5a0b6c3176 None None] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:16:03.607996 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5891833c-e0d8-4a89-a3a8-1a5a0b6c3176 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:16:03.607996 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5891833c-e0d8-4a89-a3a8-1a5a0b6c3176 None None] PATH_INFO: `/` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:16:03.609326 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 653/1312] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 12:16:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:16:03.614504 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12969b63-c497-4fe4-8f96-75657e4884ed None None] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:16:03.614955 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12969b63-c497-4fe4-8f96-75657e4884ed None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:16:03.615062 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12969b63-c497-4fe4-8f96-75657e4884ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:16:03.683368 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.auth.core [None req-12969b63-c497-4fe4-8f96-75657e4884ed None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81449) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:16:03.684551 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-12969b63-c497-4fe4-8f96-75657e4884ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:16:03.779036 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 660/1313] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 12:16:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:16:04.077048 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae9273c-74fa-4f9a-a47e-a682cf0dc62c None None] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:16:04.077332 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae9273c-74fa-4f9a-a47e-a682cf0dc62c None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:16:04.077602 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae9273c-74fa-4f9a-a47e-a682cf0dc62c None None] PATH_INFO: `/` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:16:04.078934 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 654/1314] 10.222.0.15 () {54 vars in 965 bytes} [Thu Sep 15 12:16:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:16:04.085365 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a378df-e863-498f-962c-11b0d9ccea62 None None] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:16:04.085742 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a378df-e863-498f-962c-11b0d9ccea62 None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:16:04.086082 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a378df-e863-498f-962c-11b0d9ccea62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:16:04.098041 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:16:04.165309 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.auth.core [None req-a0a378df-e863-498f-962c-11b0d9ccea62 None None] MFA Rules not processed for user `bfbb4d6809064f9cb9e06626ed7d6814`. Rule list: `[]` (Enabled: `True`). {{(pid=81449) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:16:04.166430 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-a0a378df-e863-498f-962c-11b0d9ccea62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:16:04.242248 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 661/1315] 10.222.0.15 () {58 vars in 1062 bytes} [Thu Sep 15 12:16:04 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:16:04.251491 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae9273c-74fa-4f9a-a47e-a682cf0dc62c None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:16:04.253042 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-dae9273c-74fa-4f9a-a47e-a682cf0dc62c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:16:04.356218 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae9273c-74fa-4f9a-a47e-a682cf0dc62c None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:16:04.430140 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:16:04.432531 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77369f62-6cff-47d1-b91b-4c973958ad2b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfbb4d6809064f9cb9e06626ed7d6814', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '540a32ef88764e8c8d9201ae22f6ddaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:16:04.433354 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77369f62-6cff-47d1-b91b-4c973958ad2b service neutron] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:16:04.433694 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77369f62-6cff-47d1-b91b-4c973958ad2b service neutron] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:16:04.434161 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77369f62-6cff-47d1-b91b-4c973958ad2b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:16:04.435982 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-77369f62-6cff-47d1-b91b-4c973958ad2b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:16:04.544006 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77369f62-6cff-47d1-b91b-4c973958ad2b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:16:04.545483 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77369f62-6cff-47d1-b91b-4c973958ad2b service neutron] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:16:04.563380 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 655/1316] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 12:16:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 12:16:07.258262 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840650d5-fe92-419d-8142-70df16f1b274 None None] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:16:07.258773 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840650d5-fe92-419d-8142-70df16f1b274 None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:16:07.258773 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840650d5-fe92-419d-8142-70df16f1b274 None None] PATH_INFO: `/` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:16:07.260050 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 662/1317] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 12:16:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:16:07.265635 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3aad56f-cdb5-4bd9-b984-69f4bfd387f3 None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:16:07.266081 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3aad56f-cdb5-4bd9-b984-69f4bfd387f3 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:16:07.266176 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3aad56f-cdb5-4bd9-b984-69f4bfd387f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:16:07.330797 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-f3aad56f-cdb5-4bd9-b984-69f4bfd387f3 None None] MFA Rules not processed for user `6147f6de110b41cd99def2cbeb937ad1`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:16:07.332042 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-f3aad56f-cdb5-4bd9-b984-69f4bfd387f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:16:07.383537 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:16:07.427211 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 656/1318] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 12:16:07 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:16:07.711786 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840650d5-fe92-419d-8142-70df16f1b274 None None] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:16:07.713318 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-840650d5-fe92-419d-8142-70df16f1b274 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:16:07.815650 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840650d5-fe92-419d-8142-70df16f1b274 None None] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:16:07.818473 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:16:07.888064 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb8170d-0841-45e4-a8a7-9e6aec129384 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfbb4d6809064f9cb9e06626ed7d6814', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '540a32ef88764e8c8d9201ae22f6ddaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:16:07.888665 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb8170d-0841-45e4-a8a7-9e6aec129384 service neutron] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:16:07.888965 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb8170d-0841-45e4-a8a7-9e6aec129384 service neutron] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:16:07.889236 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb8170d-0841-45e4-a8a7-9e6aec129384 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:16:07.890812 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-5eb8170d-0841-45e4-a8a7-9e6aec129384 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:16:07.996434 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb8170d-0841-45e4-a8a7-9e6aec129384 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:16:07.997902 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb8170d-0841-45e4-a8a7-9e6aec129384 service neutron] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:16:08.015629 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 663/1319] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 12:16:07 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 12:16:10.665024 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5836d52f-ab97-42ac-b15e-709d0ba827f5 None None] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:16:10.665590 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5836d52f-ab97-42ac-b15e-709d0ba827f5 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:16:10.665590 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5836d52f-ab97-42ac-b15e-709d0ba827f5 None None] PATH_INFO: `/` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:16:10.666893 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 657/1320] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 12:16:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:16:10.672120 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329c049c-0b54-47ca-9a25-945f15c26ff0 None None] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:16:10.672542 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329c049c-0b54-47ca-9a25-945f15c26ff0 None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:16:10.672656 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329c049c-0b54-47ca-9a25-945f15c26ff0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:16:10.712338 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:16:10.738033 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.auth.core [None req-329c049c-0b54-47ca-9a25-945f15c26ff0 None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81449) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:16:10.739212 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-329c049c-0b54-47ca-9a25-945f15c26ff0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:16:10.832971 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 664/1321] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 12:16:10 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:16:11.119866 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5836d52f-ab97-42ac-b15e-709d0ba827f5 None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:16:11.121428 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-5836d52f-ab97-42ac-b15e-709d0ba827f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:16:11.231070 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5836d52f-ab97-42ac-b15e-709d0ba827f5 None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:16:11.302901 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e58ba28-3678-41d6-ad23-16a7098c166a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfbb4d6809064f9cb9e06626ed7d6814', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '540a32ef88764e8c8d9201ae22f6ddaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:16:11.303510 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e58ba28-3678-41d6-ad23-16a7098c166a service neutron] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:16:11.303791 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e58ba28-3678-41d6-ad23-16a7098c166a service neutron] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:16:11.304061 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e58ba28-3678-41d6-ad23-16a7098c166a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:16:11.305687 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-1e58ba28-3678-41d6-ad23-16a7098c166a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:16:11.312477 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:16:11.413095 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e58ba28-3678-41d6-ad23-16a7098c166a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:16:11.414515 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e58ba28-3678-41d6-ad23-16a7098c166a service neutron] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:16:11.432105 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 658/1322] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 12:16:11 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 12:16:14.230334 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0991a55f-631a-47ef-a093-26f924d29678 None None] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:16:14.230859 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0991a55f-631a-47ef-a093-26f924d29678 None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:16:14.230859 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0991a55f-631a-47ef-a093-26f924d29678 None None] PATH_INFO: `/` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:16:14.232131 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 665/1323] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 12:16:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:16:14.237544 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b363bf5-00b8-452e-a876-63d5f5b73396 None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:16:14.237986 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b363bf5-00b8-452e-a876-63d5f5b73396 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:16:14.238082 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b363bf5-00b8-452e-a876-63d5f5b73396 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:16:14.310404 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-7b363bf5-00b8-452e-a876-63d5f5b73396 None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:16:14.311652 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-7b363bf5-00b8-452e-a876-63d5f5b73396 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:16:14.374952 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:16:14.416101 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 659/1324] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 12:16:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:16:14.695381 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0991a55f-631a-47ef-a093-26f924d29678 None None] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:16:14.696927 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-0991a55f-631a-47ef-a093-26f924d29678 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:16:14.802622 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0991a55f-631a-47ef-a093-26f924d29678 None None] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:16:14.814800 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:16:14.875027 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae12a3e9-f5fe-45a8-8df0-b96f439edf03 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfbb4d6809064f9cb9e06626ed7d6814', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '540a32ef88764e8c8d9201ae22f6ddaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:16:14.875624 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae12a3e9-f5fe-45a8-8df0-b96f439edf03 service neutron] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:16:14.875897 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae12a3e9-f5fe-45a8-8df0-b96f439edf03 service neutron] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:16:14.876164 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae12a3e9-f5fe-45a8-8df0-b96f439edf03 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:16:14.878385 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-ae12a3e9-f5fe-45a8-8df0-b96f439edf03 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:16:14.987495 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae12a3e9-f5fe-45a8-8df0-b96f439edf03 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:16:14.988992 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae12a3e9-f5fe-45a8-8df0-b96f439edf03 service neutron] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:16:15.007264 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 666/1325] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 12:16:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 12:16:17.227016 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4129ee75-2fb8-492d-836a-d1a35ef04cad None None] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:16:17.227540 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4129ee75-2fb8-492d-836a-d1a35ef04cad None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:16:17.227540 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4129ee75-2fb8-492d-836a-d1a35ef04cad None None] PATH_INFO: `/` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:16:17.229008 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 660/1326] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 12:16:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:16:17.234054 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692b03ce-79a0-458c-858f-3e19c77e0e87 None None] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:16:17.234502 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692b03ce-79a0-458c-858f-3e19c77e0e87 None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:16:17.234601 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692b03ce-79a0-458c-858f-3e19c77e0e87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:16:17.282005 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:16:17.301053 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.auth.core [None req-692b03ce-79a0-458c-858f-3e19c77e0e87 None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81449) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:16:17.302257 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-692b03ce-79a0-458c-858f-3e19c77e0e87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:16:17.402857 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 667/1327] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 12:16:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:16:17.686083 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4129ee75-2fb8-492d-836a-d1a35ef04cad None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:16:17.687614 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-4129ee75-2fb8-492d-836a-d1a35ef04cad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:16:17.791681 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4129ee75-2fb8-492d-836a-d1a35ef04cad None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:16:17.865237 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2228f4ca-a9c1-47a7-bb7e-e7011c429c1b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfbb4d6809064f9cb9e06626ed7d6814', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '540a32ef88764e8c8d9201ae22f6ddaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:16:17.865849 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2228f4ca-a9c1-47a7-bb7e-e7011c429c1b service neutron] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:16:17.866127 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2228f4ca-a9c1-47a7-bb7e-e7011c429c1b service neutron] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:16:17.866393 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2228f4ca-a9c1-47a7-bb7e-e7011c429c1b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:16:17.867968 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-2228f4ca-a9c1-47a7-bb7e-e7011c429c1b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:16:17.885444 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:16:17.979874 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2228f4ca-a9c1-47a7-bb7e-e7011c429c1b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:16:17.981332 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2228f4ca-a9c1-47a7-bb7e-e7011c429c1b service neutron] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:16:17.999473 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 661/1328] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 12:16:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 12:16:20.191959 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4021e4d3-c10a-46a8-ab45-66253cd68411 None None] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:16:20.192467 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4021e4d3-c10a-46a8-ab45-66253cd68411 None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:16:20.192562 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4021e4d3-c10a-46a8-ab45-66253cd68411 None None] PATH_INFO: `/` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:16:20.193900 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 668/1329] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 12:16:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:16:20.199058 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e464a1-a81a-46b2-a3e8-c2ea7afc82b0 None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:16:20.199472 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e464a1-a81a-46b2-a3e8-c2ea7afc82b0 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:16:20.199582 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e464a1-a81a-46b2-a3e8-c2ea7afc82b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:16:20.268125 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-61e464a1-a81a-46b2-a3e8-c2ea7afc82b0 None None] MFA Rules not processed for user `6147f6de110b41cd99def2cbeb937ad1`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:16:20.269450 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-61e464a1-a81a-46b2-a3e8-c2ea7afc82b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:16:20.335130 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:16:20.368914 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 662/1330] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 12:16:20 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:16:20.659561 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4021e4d3-c10a-46a8-ab45-66253cd68411 None None] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:16:20.661148 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-4021e4d3-c10a-46a8-ab45-66253cd68411 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:16:20.769679 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4021e4d3-c10a-46a8-ab45-66253cd68411 None None] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:16:20.791747 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:16:20.849090 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d44fe1-bfe1-4e28-a075-fb4ea61abcde service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfbb4d6809064f9cb9e06626ed7d6814', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '540a32ef88764e8c8d9201ae22f6ddaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:16:20.849702 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d44fe1-bfe1-4e28-a075-fb4ea61abcde service neutron] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:16:20.849978 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d44fe1-bfe1-4e28-a075-fb4ea61abcde service neutron] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:16:20.850244 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d44fe1-bfe1-4e28-a075-fb4ea61abcde service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:16:20.851833 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-c7d44fe1-bfe1-4e28-a075-fb4ea61abcde service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:16:20.961935 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d44fe1-bfe1-4e28-a075-fb4ea61abcde service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:16:20.963379 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d44fe1-bfe1-4e28-a075-fb4ea61abcde service neutron] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:16:20.982634 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 669/1331] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 12:16:20 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 12:16:23.544178 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffc3d93-b385-4462-be2d-a0214cb02893 None None] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:16:23.544669 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffc3d93-b385-4462-be2d-a0214cb02893 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:16:23.544669 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ffc3d93-b385-4462-be2d-a0214cb02893 None None] PATH_INFO: `/` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:16:23.546034 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 663/1332] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 12:16:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:16:23.551864 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd82fcf-26c8-46e4-80f7-02b589283f12 None None] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:16:23.552266 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd82fcf-26c8-46e4-80f7-02b589283f12 None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:16:23.552417 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd82fcf-26c8-46e4-80f7-02b589283f12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:16:23.615999 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:16:23.618812 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.auth.core [None req-4fd82fcf-26c8-46e4-80f7-02b589283f12 None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81449) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:16:23.620459 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-4fd82fcf-26c8-46e4-80f7-02b589283f12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:16:23.717053 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 670/1333] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 12:16:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:16:24.003872 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ffc3d93-b385-4462-be2d-a0214cb02893 None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:16:24.005433 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-4ffc3d93-b385-4462-be2d-a0214cb02893 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:16:24.106938 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ffc3d93-b385-4462-be2d-a0214cb02893 None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:16:24.178512 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77fd7c01-5ad0-4bf9-a87c-74fa3282f536 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfbb4d6809064f9cb9e06626ed7d6814', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '540a32ef88764e8c8d9201ae22f6ddaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:16:24.179115 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fd7c01-5ad0-4bf9-a87c-74fa3282f536 service neutron] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:16:24.179390 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fd7c01-5ad0-4bf9-a87c-74fa3282f536 service neutron] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:16:24.179657 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77fd7c01-5ad0-4bf9-a87c-74fa3282f536 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:16:24.181259 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-77fd7c01-5ad0-4bf9-a87c-74fa3282f536 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:16:24.208009 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:16:24.288801 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77fd7c01-5ad0-4bf9-a87c-74fa3282f536 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:16:24.290148 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77fd7c01-5ad0-4bf9-a87c-74fa3282f536 service neutron] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:16:24.308008 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 664/1334] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 12:16:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 12:16:26.477920 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e2a74f-a986-4a0b-8253-194807526a0a None None] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:16:26.478835 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e2a74f-a986-4a0b-8253-194807526a0a None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:16:26.478835 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e2a74f-a986-4a0b-8253-194807526a0a None None] PATH_INFO: `/` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:16:26.479812 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 671/1335] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 12:16:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:16:26.485002 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ccba4a-55e6-4e28-868c-9cc5ace5d197 None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:16:26.485408 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ccba4a-55e6-4e28-868c-9cc5ace5d197 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:16:26.485546 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ccba4a-55e6-4e28-868c-9cc5ace5d197 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:16:26.552817 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-58ccba4a-55e6-4e28-868c-9cc5ace5d197 None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:16:26.554047 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-58ccba4a-55e6-4e28-868c-9cc5ace5d197 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:16:26.630013 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:16:26.653640 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 665/1336] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 12:16:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:16:26.936013 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e2a74f-a986-4a0b-8253-194807526a0a None None] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:16:26.937570 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-54e2a74f-a986-4a0b-8253-194807526a0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:16:27.040318 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e2a74f-a986-4a0b-8253-194807526a0a None None] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:16:27.069669 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:16:27.113923 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6dec5a0-fe38-44d1-9d37-867672bf68b8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfbb4d6809064f9cb9e06626ed7d6814', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '540a32ef88764e8c8d9201ae22f6ddaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:16:27.114523 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6dec5a0-fe38-44d1-9d37-867672bf68b8 service neutron] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:16:27.114796 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6dec5a0-fe38-44d1-9d37-867672bf68b8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:16:27.115059 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6dec5a0-fe38-44d1-9d37-867672bf68b8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:16:27.116617 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-d6dec5a0-fe38-44d1-9d37-867672bf68b8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:16:27.223509 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6dec5a0-fe38-44d1-9d37-867672bf68b8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:16:27.224963 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6dec5a0-fe38-44d1-9d37-867672bf68b8 service neutron] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:16:27.243104 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 672/1337] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 12:16:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 12:16:29.838300 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37fbbb1-68fa-48e3-a550-18e980e05ecc None None] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:16:29.838878 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37fbbb1-68fa-48e3-a550-18e980e05ecc None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:16:29.838878 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37fbbb1-68fa-48e3-a550-18e980e05ecc None None] PATH_INFO: `/` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:16:29.840164 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 666/1338] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 12:16:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:16:29.845461 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96a1927-5f40-4535-a2d9-5c0aad2117c8 None None] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:16:29.845889 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96a1927-5f40-4535-a2d9-5c0aad2117c8 None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:16:29.845994 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96a1927-5f40-4535-a2d9-5c0aad2117c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:16:29.910684 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.auth.core [None req-c96a1927-5f40-4535-a2d9-5c0aad2117c8 None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81449) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:16:29.911864 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-c96a1927-5f40-4535-a2d9-5c0aad2117c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:16:29.919923 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:16:30.006845 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 673/1339] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 12:16:29 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:16:30.284906 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37fbbb1-68fa-48e3-a550-18e980e05ecc None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:16:30.286442 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-b37fbbb1-68fa-48e3-a550-18e980e05ecc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:16:30.387926 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37fbbb1-68fa-48e3-a550-18e980e05ecc None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:16:30.459409 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6384b677-51c9-4d0d-a9e1-0243c7c8ea22 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfbb4d6809064f9cb9e06626ed7d6814', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '540a32ef88764e8c8d9201ae22f6ddaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:16:30.460014 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6384b677-51c9-4d0d-a9e1-0243c7c8ea22 service neutron] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:16:30.460293 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6384b677-51c9-4d0d-a9e1-0243c7c8ea22 service neutron] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:16:30.460563 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6384b677-51c9-4d0d-a9e1-0243c7c8ea22 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:16:30.462300 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-6384b677-51c9-4d0d-a9e1-0243c7c8ea22 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:16:30.498085 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:16:30.570781 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6384b677-51c9-4d0d-a9e1-0243c7c8ea22 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:16:30.572191 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6384b677-51c9-4d0d-a9e1-0243c7c8ea22 service neutron] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:16:30.590362 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 667/1340] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 12:16:30 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 12:16:34.708735 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8049c2a-91e4-476d-9e47-b0aefb371431 None None] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:16:34.709451 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8049c2a-91e4-476d-9e47-b0aefb371431 None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:16:34.709771 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8049c2a-91e4-476d-9e47-b0aefb371431 None None] PATH_INFO: `/` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:16:34.711294 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 674/1341] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 12:16:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:16:34.716834 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdd4057-a465-4546-b78e-831867bbf6a2 None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:16:34.717283 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdd4057-a465-4546-b78e-831867bbf6a2 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:16:34.717916 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdd4057-a465-4546-b78e-831867bbf6a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:16:34.789918 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-3fdd4057-a465-4546-b78e-831867bbf6a2 None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:16:34.791160 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-3fdd4057-a465-4546-b78e-831867bbf6a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:16:34.882975 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:16:34.898247 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 668/1342] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 12:16:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 12:16:35.179209 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8049c2a-91e4-476d-9e47-b0aefb371431 None None] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:16:35.180879 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-d8049c2a-91e4-476d-9e47-b0aefb371431 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:16:35.284573 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8049c2a-91e4-476d-9e47-b0aefb371431 None None] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:16:35.321843 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:16:35.359814 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b7bd09-4759-4e12-a59d-14b3484e4a40 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfbb4d6809064f9cb9e06626ed7d6814', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '540a32ef88764e8c8d9201ae22f6ddaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:16:35.360448 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b7bd09-4759-4e12-a59d-14b3484e4a40 service neutron] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:16:35.360726 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b7bd09-4759-4e12-a59d-14b3484e4a40 service neutron] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:16:35.361026 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b7bd09-4759-4e12-a59d-14b3484e4a40 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:16:35.362606 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-e0b7bd09-4759-4e12-a59d-14b3484e4a40 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:16:35.470818 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b7bd09-4759-4e12-a59d-14b3484e4a40 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:16:35.472260 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b7bd09-4759-4e12-a59d-14b3484e4a40 service neutron] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:16:35.491940 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 675/1343] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 12:16:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 12:16:38.255179 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0a46b9-f2e9-48e5-ad43-9ac3483c180d None None] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:16:38.255714 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0a46b9-f2e9-48e5-ad43-9ac3483c180d None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:16:38.255831 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0a46b9-f2e9-48e5-ad43-9ac3483c180d None None] PATH_INFO: `/` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:16:38.257161 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 669/1344] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 12:16:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:16:38.262424 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365eb822-98a1-406d-bdfd-cf781958e619 None None] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:16:38.262878 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365eb822-98a1-406d-bdfd-cf781958e619 None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:16:38.263086 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365eb822-98a1-406d-bdfd-cf781958e619 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:16:38.329097 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.auth.core [None req-365eb822-98a1-406d-bdfd-cf781958e619 None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81449) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:16:38.330290 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-365eb822-98a1-406d-bdfd-cf781958e619 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:16:38.349058 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:16:38.430693 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 676/1345] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 12:16:38 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:16:38.712845 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0a46b9-f2e9-48e5-ad43-9ac3483c180d None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:16:38.714357 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-4d0a46b9-f2e9-48e5-ad43-9ac3483c180d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:16:38.824095 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0a46b9-f2e9-48e5-ad43-9ac3483c180d None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:16:38.900495 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf99a4d6-36df-4746-8058-aa074ba0de91 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfbb4d6809064f9cb9e06626ed7d6814', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '540a32ef88764e8c8d9201ae22f6ddaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:16:38.901131 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf99a4d6-36df-4746-8058-aa074ba0de91 service neutron] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:16:38.901411 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf99a4d6-36df-4746-8058-aa074ba0de91 service neutron] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:16:38.901682 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf99a4d6-36df-4746-8058-aa074ba0de91 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:16:38.903265 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-cf99a4d6-36df-4746-8058-aa074ba0de91 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:16:38.947336 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:16:39.017608 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf99a4d6-36df-4746-8058-aa074ba0de91 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:16:39.019152 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf99a4d6-36df-4746-8058-aa074ba0de91 service neutron] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:16:39.039389 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 670/1346] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 12:16:38 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 12:16:43.111368 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5177eca4-33d3-4fbc-be7d-f704a2e1d420 None None] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:16:43.112320 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5177eca4-33d3-4fbc-be7d-f704a2e1d420 None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:16:43.112789 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5177eca4-33d3-4fbc-be7d-f704a2e1d420 None None] PATH_INFO: `/` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:16:43.114743 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 677/1347] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 12:16:43 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:16:43.119671 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8ddf46-eed3-4080-8cf9-0905cd59cb67 None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:16:43.120089 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8ddf46-eed3-4080-8cf9-0905cd59cb67 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:16:43.120209 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8ddf46-eed3-4080-8cf9-0905cd59cb67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:16:43.184805 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-7f8ddf46-eed3-4080-8cf9-0905cd59cb67 None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:16:43.186002 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-7f8ddf46-eed3-4080-8cf9-0905cd59cb67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:16:43.277014 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:16:43.279350 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 671/1348] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 12:16:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:16:43.563989 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5177eca4-33d3-4fbc-be7d-f704a2e1d420 None None] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:16:43.565730 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-5177eca4-33d3-4fbc-be7d-f704a2e1d420 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:16:43.670825 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5177eca4-33d3-4fbc-be7d-f704a2e1d420 None None] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:16:43.718626 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:16:43.744688 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8933e2ac-fff4-4b9a-83ae-f16bad6321c2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfbb4d6809064f9cb9e06626ed7d6814', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '540a32ef88764e8c8d9201ae22f6ddaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:16:43.745326 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8933e2ac-fff4-4b9a-83ae-f16bad6321c2 service neutron] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:16:43.745603 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8933e2ac-fff4-4b9a-83ae-f16bad6321c2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:16:43.745876 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8933e2ac-fff4-4b9a-83ae-f16bad6321c2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:16:43.747501 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-8933e2ac-fff4-4b9a-83ae-f16bad6321c2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:16:43.862271 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8933e2ac-fff4-4b9a-83ae-f16bad6321c2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:16:43.863694 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8933e2ac-fff4-4b9a-83ae-f16bad6321c2 service neutron] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:16:43.881606 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 678/1349] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 12:16:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 12:16:48.816785 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ddfa71-a68a-4146-ae75-e835279421cb None None] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:16:48.816785 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ddfa71-a68a-4146-ae75-e835279421cb None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:16:48.817393 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ddfa71-a68a-4146-ae75-e835279421cb None None] PATH_INFO: `/` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:16:48.819259 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 672/1350] 10.222.0.15 () {54 vars in 921 bytes} [Thu Sep 15 12:16:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:16:48.826071 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1623707e-ab96-4707-a48d-247714563c67 None None] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:16:48.826530 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1623707e-ab96-4707-a48d-247714563c67 None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:16:48.826904 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1623707e-ab96-4707-a48d-247714563c67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:16:48.927479 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.auth.core [None req-1623707e-ab96-4707-a48d-247714563c67 None None] MFA Rules not processed for user `51f64a058bd5440cb5268f228aa6e392`. Rule list: `[]` (Enabled: `True`). {{(pid=81449) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:16:48.928898 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-1623707e-ab96-4707-a48d-247714563c67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:16:48.950754 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:16:49.133946 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 679/1351] 10.222.0.15 () {58 vars in 1018 bytes} [Thu Sep 15 12:16:48 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 309 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:16:53.241653 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56da2633-b1ff-41bf-b6df-58c10a0f681e None None] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:16:53.242203 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56da2633-b1ff-41bf-b6df-58c10a0f681e None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:16:53.242399 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56da2633-b1ff-41bf-b6df-58c10a0f681e None None] PATH_INFO: `/` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:16:53.244224 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 673/1352] 10.222.0.15 () {54 vars in 921 bytes} [Thu Sep 15 12:16:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:16:53.249142 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290b36e0-02e1-4eda-8a9e-668ecf51b140 None None] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:16:53.249629 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290b36e0-02e1-4eda-8a9e-668ecf51b140 None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:16:53.250542 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290b36e0-02e1-4eda-8a9e-668ecf51b140 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:16:53.331177 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.auth.core [None req-290b36e0-02e1-4eda-8a9e-668ecf51b140 None None] MFA Rules not processed for user `51f64a058bd5440cb5268f228aa6e392`. Rule list: `[]` (Enabled: `True`). {{(pid=81449) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:16:53.332508 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-290b36e0-02e1-4eda-8a9e-668ecf51b140 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:16:53.407432 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:16:53.446240 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 680/1353] 10.222.0.15 () {58 vars in 1018 bytes} [Thu Sep 15 12:16:53 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:16:54.828206 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a67ae3c-34d4-4be2-a521-5a6e84ec71ba None None] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:16:54.828893 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a67ae3c-34d4-4be2-a521-5a6e84ec71ba None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:16:54.828893 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a67ae3c-34d4-4be2-a521-5a6e84ec71ba None None] PATH_INFO: `/` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:16:54.830109 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 674/1354] 10.222.0.15 () {54 vars in 921 bytes} [Thu Sep 15 12:16:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:16:54.835050 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e4dee0-84ef-4345-b6f8-7a79f653f8ce None None] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:16:54.835494 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e4dee0-84ef-4345-b6f8-7a79f653f8ce None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:16:54.835599 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e4dee0-84ef-4345-b6f8-7a79f653f8ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:16:54.911426 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.auth.core [None req-76e4dee0-84ef-4345-b6f8-7a79f653f8ce None None] MFA Rules not processed for user `51f64a058bd5440cb5268f228aa6e392`. Rule list: `[]` (Enabled: `True`). {{(pid=81449) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:16:54.912635 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-76e4dee0-84ef-4345-b6f8-7a79f653f8ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:16:55.022410 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 681/1355] 10.222.0.15 () {58 vars in 1018 bytes} [Thu Sep 15 12:16:54 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:16:58.545623 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c942c08c-faf0-40ef-bd61-483a63f02282 None None] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:16:58.546191 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c942c08c-faf0-40ef-bd61-483a63f02282 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:16:58.546191 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c942c08c-faf0-40ef-bd61-483a63f02282 None None] PATH_INFO: `/` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:16:58.547474 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 675/1356] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 12:16:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:16:58.553196 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7c86fe-44e3-4463-b2d5-b1a1a425a22d None None] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:16:58.553679 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7c86fe-44e3-4463-b2d5-b1a1a425a22d None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:16:58.553776 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7c86fe-44e3-4463-b2d5-b1a1a425a22d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:16:58.594549 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:16:58.630662 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.auth.core [None req-ce7c86fe-44e3-4463-b2d5-b1a1a425a22d None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81449) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:16:58.631798 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-ce7c86fe-44e3-4463-b2d5-b1a1a425a22d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:16:58.733865 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 682/1357] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 12:16:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:16:59.043667 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c40d9d2-ae28-4b1f-ae66-a7b1fd33c935 None None] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:16:59.044141 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c40d9d2-ae28-4b1f-ae66-a7b1fd33c935 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:16:59.044581 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c40d9d2-ae28-4b1f-ae66-a7b1fd33c935 None None] PATH_INFO: `/` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:16:59.046905 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 676/1358] 10.222.0.15 () {54 vars in 962 bytes} [Thu Sep 15 12:16:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:16:59.056644 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb0d06f-a7b5-45b1-86e0-0c53fbabca2c None None] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:16:59.057175 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb0d06f-a7b5-45b1-86e0-0c53fbabca2c None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:16:59.057643 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb0d06f-a7b5-45b1-86e0-0c53fbabca2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:16:59.148365 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.auth.core [None req-feb0d06f-a7b5-45b1-86e0-0c53fbabca2c None None] MFA Rules not processed for user `eface89e261840a2abb4206be26455c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81449) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:16:59.149983 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-feb0d06f-a7b5-45b1-86e0-0c53fbabca2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:16:59.205388 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:16:59.277470 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 683/1359] 10.222.0.15 () {58 vars in 1059 bytes} [Thu Sep 15 12:16:59 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:16:59.285242 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c40d9d2-ae28-4b1f-ae66-a7b1fd33c935 None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:16:59.286781 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-9c40d9d2-ae28-4b1f-ae66-a7b1fd33c935 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:16:59.449596 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c40d9d2-ae28-4b1f-ae66-a7b1fd33c935 None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:16:59.532181 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:16:59.574781 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0533ea8-2351-49c1-b6b1-2dc679120b4c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eface89e261840a2abb4206be26455c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '540a32ef88764e8c8d9201ae22f6ddaf', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:16:59.575539 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0533ea8-2351-49c1-b6b1-2dc679120b4c service nova] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:16:59.575921 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0533ea8-2351-49c1-b6b1-2dc679120b4c service nova] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:16:59.576316 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0533ea8-2351-49c1-b6b1-2dc679120b4c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:16:59.578073 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-e0533ea8-2351-49c1-b6b1-2dc679120b4c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:16:59.682131 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0533ea8-2351-49c1-b6b1-2dc679120b4c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:16:59.683893 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0533ea8-2351-49c1-b6b1-2dc679120b4c service nova] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:16:59.704792 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 677/1360] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 12:16:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 420 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 12:17:02.070938 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608858c9-207e-4f80-8881-4cc0bab38a5b None None] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:17:02.071584 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608858c9-207e-4f80-8881-4cc0bab38a5b None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:17:02.071584 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608858c9-207e-4f80-8881-4cc0bab38a5b None None] PATH_INFO: `/` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:17:02.076581 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 684/1361] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 12:17:02 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:17:02.078411 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479c47ef-68c4-4580-846c-4a3c7fd622bc None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:17:02.078985 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479c47ef-68c4-4580-846c-4a3c7fd622bc None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:17:02.078985 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479c47ef-68c4-4580-846c-4a3c7fd622bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:17:02.152589 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-479c47ef-68c4-4580-846c-4a3c7fd622bc None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:17:02.153830 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-479c47ef-68c4-4580-846c-4a3c7fd622bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:17:02.197678 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:17:02.259218 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 678/1362] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 12:17:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 12:17:02.553288 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608858c9-207e-4f80-8881-4cc0bab38a5b None None] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:17:02.554924 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-608858c9-207e-4f80-8881-4cc0bab38a5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:17:02.732307 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608858c9-207e-4f80-8881-4cc0bab38a5b None None] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:17:02.745080 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:17:02.881936 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-546b0375-a4bb-48e6-9cb3-2b27e3345ef1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eface89e261840a2abb4206be26455c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '540a32ef88764e8c8d9201ae22f6ddaf', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:17:02.882547 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546b0375-a4bb-48e6-9cb3-2b27e3345ef1 service nova] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:17:02.882822 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546b0375-a4bb-48e6-9cb3-2b27e3345ef1 service nova] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:17:02.883092 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546b0375-a4bb-48e6-9cb3-2b27e3345ef1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:17:02.884693 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-546b0375-a4bb-48e6-9cb3-2b27e3345ef1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:17:02.979899 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546b0375-a4bb-48e6-9cb3-2b27e3345ef1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:17:02.981584 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-546b0375-a4bb-48e6-9cb3-2b27e3345ef1 service nova] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:17:03.000837 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 685/1363] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 12:17:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 448 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 12:17:05.460059 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7eb93c-c0b6-4fe1-ad34-d60dfba96a9f None None] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:17:05.460599 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7eb93c-c0b6-4fe1-ad34-d60dfba96a9f None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:17:05.460599 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7eb93c-c0b6-4fe1-ad34-d60dfba96a9f None None] PATH_INFO: `/` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:17:05.462224 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 679/1364] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 12:17:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:17:05.467858 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2572ba-711b-40eb-8424-df1d3c8975d1 None None] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:17:05.468509 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2572ba-711b-40eb-8424-df1d3c8975d1 None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:17:05.468681 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2572ba-711b-40eb-8424-df1d3c8975d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:17:05.484514 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:17:05.537919 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.auth.core [None req-8e2572ba-711b-40eb-8424-df1d3c8975d1 None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81449) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:17:05.539022 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-8e2572ba-711b-40eb-8424-df1d3c8975d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:17:05.640623 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 686/1365] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 12:17:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:17:05.931253 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c51cf7-b22c-4665-85cb-8491d4156034 None None] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:17:05.931762 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c51cf7-b22c-4665-85cb-8491d4156034 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:17:05.932132 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c51cf7-b22c-4665-85cb-8491d4156034 None None] PATH_INFO: `/` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:17:05.933604 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 680/1366] 10.222.0.15 () {54 vars in 962 bytes} [Thu Sep 15 12:17:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:17:05.939180 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ddac15-681c-445c-b971-27b1121946ed None None] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:17:05.939539 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ddac15-681c-445c-b971-27b1121946ed None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:17:05.939885 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ddac15-681c-445c-b971-27b1121946ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:17:06.019397 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.auth.core [None req-78ddac15-681c-445c-b971-27b1121946ed None None] MFA Rules not processed for user `eface89e261840a2abb4206be26455c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81449) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:17:06.020591 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-78ddac15-681c-445c-b971-27b1121946ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:17:06.057949 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:17:06.144335 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 687/1367] 10.222.0.15 () {58 vars in 1059 bytes} [Thu Sep 15 12:17:05 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:17:06.151901 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c51cf7-b22c-4665-85cb-8491d4156034 None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:17:06.153441 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-32c51cf7-b22c-4665-85cb-8491d4156034 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:17:06.309288 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c51cf7-b22c-4665-85cb-8491d4156034 None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:17:06.331993 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:17:06.430389 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b90a495-19a0-48fe-a01f-65e39bde8418 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eface89e261840a2abb4206be26455c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '540a32ef88764e8c8d9201ae22f6ddaf', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:17:06.431009 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b90a495-19a0-48fe-a01f-65e39bde8418 service nova] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:17:06.431309 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b90a495-19a0-48fe-a01f-65e39bde8418 service nova] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:17:06.431565 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b90a495-19a0-48fe-a01f-65e39bde8418 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:17:06.433210 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-0b90a495-19a0-48fe-a01f-65e39bde8418 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:17:06.529238 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b90a495-19a0-48fe-a01f-65e39bde8418 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:17:06.531392 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b90a495-19a0-48fe-a01f-65e39bde8418 service nova] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:17:06.550157 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 681/1368] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 12:17:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 12:17:06.712348 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64727f9-8c73-4e13-89cf-4d449792ee0d None None] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:17:06.712928 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64727f9-8c73-4e13-89cf-4d449792ee0d None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:17:06.712928 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64727f9-8c73-4e13-89cf-4d449792ee0d None None] PATH_INFO: `/` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:17:06.714690 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 688/1369] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 12:17:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:17:06.721309 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f71c0b-f90d-446d-ab6a-f29bc400caec None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:17:06.721670 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f71c0b-f90d-446d-ab6a-f29bc400caec None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:17:06.721853 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f71c0b-f90d-446d-ab6a-f29bc400caec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:17:06.759707 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:17:06.801655 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-21f71c0b-f90d-446d-ab6a-f29bc400caec None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:17:06.802917 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-21f71c0b-f90d-446d-ab6a-f29bc400caec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:17:06.918363 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 682/1370] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 12:17:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:17:07.251534 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19df52b6-338b-4ac8-bba4-4e9fa3aed488 None None] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:17:07.251819 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19df52b6-338b-4ac8-bba4-4e9fa3aed488 None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:17:07.252094 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19df52b6-338b-4ac8-bba4-4e9fa3aed488 None None] PATH_INFO: `/` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:17:07.253593 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 689/1371] 10.222.0.15 () {54 vars in 964 bytes} [Thu Sep 15 12:17:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:17:07.258390 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f03737-d18d-4e38-8cfc-def6c86c265a None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:17:07.258784 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f03737-d18d-4e38-8cfc-def6c86c265a None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:17:07.259184 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f03737-d18d-4e38-8cfc-def6c86c265a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:17:07.340556 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-71f03737-d18d-4e38-8cfc-def6c86c265a None None] MFA Rules not processed for user `c8191d3ccb874529b32f8d0026db629d`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:17:07.341917 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-71f03737-d18d-4e38-8cfc-def6c86c265a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:17:07.394138 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:17:07.427105 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 683/1372] 10.222.0.15 () {58 vars in 1061 bytes} [Thu Sep 15 12:17:07 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 12:17:07.434832 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19df52b6-338b-4ac8-bba4-4e9fa3aed488 None None] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:17:07.436709 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-19df52b6-338b-4ac8-bba4-4e9fa3aed488 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:17:07.547025 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19df52b6-338b-4ac8-bba4-4e9fa3aed488 None None] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:17:07.589366 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:17:07.627766 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f008ebd2-2f7c-4377-b612-a61258a95c40 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8191d3ccb874529b32f8d0026db629d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '540a32ef88764e8c8d9201ae22f6ddaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:17:07.628380 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f008ebd2-2f7c-4377-b612-a61258a95c40 service cinder] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:17:07.628653 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f008ebd2-2f7c-4377-b612-a61258a95c40 service cinder] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:17:07.628962 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f008ebd2-2f7c-4377-b612-a61258a95c40 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:17:07.630542 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-f008ebd2-2f7c-4377-b612-a61258a95c40 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:17:07.755018 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f008ebd2-2f7c-4377-b612-a61258a95c40 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:17:07.756432 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f008ebd2-2f7c-4377-b612-a61258a95c40 service cinder] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:17:07.775839 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 690/1373] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 12:17:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 12:17:08.819500 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a296f2-b667-4ab8-a5cd-8a7a69a54a33 None None] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:17:08.820014 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a296f2-b667-4ab8-a5cd-8a7a69a54a33 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:17:08.820014 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a296f2-b667-4ab8-a5cd-8a7a69a54a33 None None] PATH_INFO: `/` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:17:08.821323 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 684/1374] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 12:17:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:17:08.826991 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a97477-63e3-483a-81a3-e5be72789e1e None None] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:17:08.827416 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a97477-63e3-483a-81a3-e5be72789e1e None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:17:08.827566 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a97477-63e3-483a-81a3-e5be72789e1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:17:08.893960 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.auth.core [None req-45a97477-63e3-483a-81a3-e5be72789e1e None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81449) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:17:08.895172 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-45a97477-63e3-483a-81a3-e5be72789e1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:17:08.919029 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:17:09.000567 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 691/1375] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 12:17:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:17:09.294376 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a296f2-b667-4ab8-a5cd-8a7a69a54a33 None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:17:09.295938 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-41a296f2-b667-4ab8-a5cd-8a7a69a54a33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:17:09.455348 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a296f2-b667-4ab8-a5cd-8a7a69a54a33 None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:17:09.508910 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:17:09.572065 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77cdd178-d692-4b47-9e8d-2c2a7d1ceb51 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eface89e261840a2abb4206be26455c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '540a32ef88764e8c8d9201ae22f6ddaf', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:17:09.572674 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cdd178-d692-4b47-9e8d-2c2a7d1ceb51 service nova] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:17:09.572976 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cdd178-d692-4b47-9e8d-2c2a7d1ceb51 service nova] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:17:09.573241 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77cdd178-d692-4b47-9e8d-2c2a7d1ceb51 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:17:09.574813 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-77cdd178-d692-4b47-9e8d-2c2a7d1ceb51 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:17:09.668688 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77cdd178-d692-4b47-9e8d-2c2a7d1ceb51 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:17:09.670853 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77cdd178-d692-4b47-9e8d-2c2a7d1ceb51 service nova] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:17:09.690957 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 685/1376] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 12:17:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 12:17:11.925105 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af17895e-e0b3-412b-8b3f-2e4ad4676331 None None] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:17:11.925671 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af17895e-e0b3-412b-8b3f-2e4ad4676331 None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:17:11.925671 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af17895e-e0b3-412b-8b3f-2e4ad4676331 None None] PATH_INFO: `/` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:17:11.927141 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 692/1377] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 12:17:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:17:11.932709 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a92e9df-c823-4549-9fb1-f2e7327f6993 None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:17:11.933544 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a92e9df-c823-4549-9fb1-f2e7327f6993 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:17:11.933852 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a92e9df-c823-4549-9fb1-f2e7327f6993 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:17:12.015950 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-9a92e9df-c823-4549-9fb1-f2e7327f6993 None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:17:12.017158 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-9a92e9df-c823-4549-9fb1-f2e7327f6993 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:17:12.026451 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:17:12.136581 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 686/1378] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 12:17:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:17:12.436730 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af17895e-e0b3-412b-8b3f-2e4ad4676331 None None] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:17:12.438353 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-af17895e-e0b3-412b-8b3f-2e4ad4676331 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:17:12.590149 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af17895e-e0b3-412b-8b3f-2e4ad4676331 None None] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:17:12.594962 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:17:12.716355 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4248d463-3184-42d7-92b3-cf0961378ad8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eface89e261840a2abb4206be26455c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '540a32ef88764e8c8d9201ae22f6ddaf', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:17:12.717565 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4248d463-3184-42d7-92b3-cf0961378ad8 service nova] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:17:12.717845 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4248d463-3184-42d7-92b3-cf0961378ad8 service nova] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:17:12.718367 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4248d463-3184-42d7-92b3-cf0961378ad8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:17:12.719953 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-4248d463-3184-42d7-92b3-cf0961378ad8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:17:12.813438 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4248d463-3184-42d7-92b3-cf0961378ad8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:17:12.815031 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4248d463-3184-42d7-92b3-cf0961378ad8 service nova] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:17:12.834013 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 693/1379] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 12:17:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 12:17:15.052040 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4fa703-ae51-4179-9a4c-66e41c47b2a1 None None] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:17:15.052594 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4fa703-ae51-4179-9a4c-66e41c47b2a1 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:17:15.052594 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4fa703-ae51-4179-9a4c-66e41c47b2a1 None None] PATH_INFO: `/` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:17:15.054081 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 687/1380] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 12:17:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:17:15.059185 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ef10a3-9ca5-44d6-b53c-b97569f0e04f None None] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:17:15.059578 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ef10a3-9ca5-44d6-b53c-b97569f0e04f None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:17:15.059741 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ef10a3-9ca5-44d6-b53c-b97569f0e04f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:17:15.071639 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:17:15.130193 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.auth.core [None req-45ef10a3-9ca5-44d6-b53c-b97569f0e04f None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81449) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:17:15.131389 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-45ef10a3-9ca5-44d6-b53c-b97569f0e04f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:17:15.253375 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 694/1381] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 12:17:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:17:15.544642 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4fa703-ae51-4179-9a4c-66e41c47b2a1 None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:17:15.546361 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-5e4fa703-ae51-4179-9a4c-66e41c47b2a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:17:15.686580 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:17:15.700831 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4fa703-ae51-4179-9a4c-66e41c47b2a1 None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:17:15.840330 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df01d060-d774-4c6b-a0f6-e95842dd0f81 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eface89e261840a2abb4206be26455c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '540a32ef88764e8c8d9201ae22f6ddaf', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:17:15.840890 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df01d060-d774-4c6b-a0f6-e95842dd0f81 service nova] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:17:15.841165 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df01d060-d774-4c6b-a0f6-e95842dd0f81 service nova] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:17:15.841433 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df01d060-d774-4c6b-a0f6-e95842dd0f81 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:17:15.843848 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-df01d060-d774-4c6b-a0f6-e95842dd0f81 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:17:15.943011 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df01d060-d774-4c6b-a0f6-e95842dd0f81 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:17:15.944616 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df01d060-d774-4c6b-a0f6-e95842dd0f81 service nova] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:17:15.955256 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:17:15.964307 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 688/1382] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 12:17:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 421 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 12:17:18.166783 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61ae68d-3fee-4d9e-9ac9-2e6a7a7cae65 None None] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:17:18.167334 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61ae68d-3fee-4d9e-9ac9-2e6a7a7cae65 None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:17:18.167441 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61ae68d-3fee-4d9e-9ac9-2e6a7a7cae65 None None] PATH_INFO: `/` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:17:18.168837 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 695/1383] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 12:17:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:17:18.175835 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333693cb-6f4a-4b36-8afe-0b822cca80a9 None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:17:18.176474 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333693cb-6f4a-4b36-8afe-0b822cca80a9 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:17:18.176474 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333693cb-6f4a-4b36-8afe-0b822cca80a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:17:18.253467 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-333693cb-6f4a-4b36-8afe-0b822cca80a9 None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:17:18.254603 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-333693cb-6f4a-4b36-8afe-0b822cca80a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:17:18.358925 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 689/1384] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 12:17:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:17:18.650576 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61ae68d-3fee-4d9e-9ac9-2e6a7a7cae65 None None] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:17:18.652165 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-a61ae68d-3fee-4d9e-9ac9-2e6a7a7cae65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:17:18.751576 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:17:18.816121 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61ae68d-3fee-4d9e-9ac9-2e6a7a7cae65 None None] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:17:18.934918 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7f6aaf-ce12-491d-934a-a2fe2c0c74f9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eface89e261840a2abb4206be26455c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '540a32ef88764e8c8d9201ae22f6ddaf', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:17:18.936051 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7f6aaf-ce12-491d-934a-a2fe2c0c74f9 service nova] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:17:18.936347 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7f6aaf-ce12-491d-934a-a2fe2c0c74f9 service nova] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:17:18.936596 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7f6aaf-ce12-491d-934a-a2fe2c0c74f9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:17:18.938241 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-9a7f6aaf-ce12-491d-934a-a2fe2c0c74f9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:17:18.998493 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:17:19.041162 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7f6aaf-ce12-491d-934a-a2fe2c0c74f9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:17:19.042804 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7f6aaf-ce12-491d-934a-a2fe2c0c74f9 service nova] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:17:19.061722 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 696/1385] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 12:17:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 12:17:21.269549 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4beb4d2-7765-4fc2-a09b-713ac513948c None None] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:17:21.270063 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4beb4d2-7765-4fc2-a09b-713ac513948c None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:17:21.270063 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4beb4d2-7765-4fc2-a09b-713ac513948c None None] PATH_INFO: `/` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:17:21.271368 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 690/1386] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 12:17:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:17:21.276831 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bbfb5a-6c31-498b-870c-92704adefe50 None None] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:17:21.277281 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bbfb5a-6c31-498b-870c-92704adefe50 None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:17:21.277395 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bbfb5a-6c31-498b-870c-92704adefe50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:17:21.347333 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.auth.core [None req-05bbfb5a-6c31-498b-870c-92704adefe50 None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81449) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:17:21.348617 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-05bbfb5a-6c31-498b-870c-92704adefe50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:17:21.447384 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 697/1387] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 12:17:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 12:17:21.765166 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4beb4d2-7765-4fc2-a09b-713ac513948c None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:17:21.766658 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-d4beb4d2-7765-4fc2-a09b-713ac513948c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:17:21.863279 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:17:21.939853 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4beb4d2-7765-4fc2-a09b-713ac513948c None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:17:22.078443 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857cc4e0-d13e-4959-9878-0a0fae10aaff service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eface89e261840a2abb4206be26455c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '540a32ef88764e8c8d9201ae22f6ddaf', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:17:22.079093 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857cc4e0-d13e-4959-9878-0a0fae10aaff service nova] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:17:22.079260 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857cc4e0-d13e-4959-9878-0a0fae10aaff service nova] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:17:22.079610 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857cc4e0-d13e-4959-9878-0a0fae10aaff service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:17:22.081296 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-857cc4e0-d13e-4959-9878-0a0fae10aaff service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:17:22.125754 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:17:22.179729 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857cc4e0-d13e-4959-9878-0a0fae10aaff service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:17:22.181385 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857cc4e0-d13e-4959-9878-0a0fae10aaff service nova] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:17:22.204833 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 691/1388] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 12:17:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 436 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 12:17:24.221403 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4832b48-ef21-408d-afa3-8860b05cbdc9 None None] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:17:24.221968 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4832b48-ef21-408d-afa3-8860b05cbdc9 None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:17:24.221968 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4832b48-ef21-408d-afa3-8860b05cbdc9 None None] PATH_INFO: `/` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:17:24.223327 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 698/1389] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 12:17:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:17:24.228611 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12097837-c4c6-4b58-b432-d7906c3f91e7 None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:17:24.229062 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12097837-c4c6-4b58-b432-d7906c3f91e7 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:17:24.229202 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12097837-c4c6-4b58-b432-d7906c3f91e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:17:24.299713 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-12097837-c4c6-4b58-b432-d7906c3f91e7 None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:17:24.300949 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-12097837-c4c6-4b58-b432-d7906c3f91e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:17:24.401928 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 692/1390] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 12:17:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:17:24.466999 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3416d9-4287-4c88-aac8-6427014c3abd None None] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:17:24.467281 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3416d9-4287-4c88-aac8-6427014c3abd None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:17:24.467816 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3416d9-4287-4c88-aac8-6427014c3abd None None] PATH_INFO: `/` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:17:24.469190 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 699/1391] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 12:17:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:17:24.474764 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19505661-7c45-49dd-9d68-1dfad93f3e6f None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:17:24.474993 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19505661-7c45-49dd-9d68-1dfad93f3e6f None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:17:24.475265 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19505661-7c45-49dd-9d68-1dfad93f3e6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:17:24.491496 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:17:24.542309 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-19505661-7c45-49dd-9d68-1dfad93f3e6f None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:17:24.543540 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-19505661-7c45-49dd-9d68-1dfad93f3e6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:17:24.647878 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 693/1392] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 12:17:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:17:24.701554 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215595ec-1605-4c9e-8a1a-0f2382e0a60f None None] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:17:24.701842 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215595ec-1605-4c9e-8a1a-0f2382e0a60f None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:17:24.702119 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215595ec-1605-4c9e-8a1a-0f2382e0a60f None None] PATH_INFO: `/` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:17:24.703484 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 700/1393] 10.222.0.15 () {54 vars in 962 bytes} [Thu Sep 15 12:17:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:17:24.708682 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072a12c6-59d3-4e8d-b18f-a6864801606b None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:17:24.709014 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072a12c6-59d3-4e8d-b18f-a6864801606b None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:17:24.709315 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072a12c6-59d3-4e8d-b18f-a6864801606b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:17:24.786594 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-072a12c6-59d3-4e8d-b18f-a6864801606b None None] MFA Rules not processed for user `3aed8f78903f44cda9e5254aa46cf2b7`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:17:24.787802 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-072a12c6-59d3-4e8d-b18f-a6864801606b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:17:24.825782 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:17:24.874199 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 694/1394] 10.222.0.15 () {58 vars in 1059 bytes} [Thu Sep 15 12:17:24 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:17:24.883021 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215595ec-1605-4c9e-8a1a-0f2382e0a60f None None] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:17:24.884549 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-215595ec-1605-4c9e-8a1a-0f2382e0a60f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:17:24.918215 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:17:24.949499 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072a12c6-59d3-4e8d-b18f-a6864801606b None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:17:24.950994 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-072a12c6-59d3-4e8d-b18f-a6864801606b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:17:24.999186 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215595ec-1605-4c9e-8a1a-0f2382e0a60f None None] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:17:25.073355 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072a12c6-59d3-4e8d-b18f-a6864801606b None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:17:25.075132 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9615cf4-07b8-485f-8366-153afdf8dd1c service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3aed8f78903f44cda9e5254aa46cf2b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '540a32ef88764e8c8d9201ae22f6ddaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:17:25.075931 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9615cf4-07b8-485f-8366-153afdf8dd1c service glance] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:17:25.076212 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9615cf4-07b8-485f-8366-153afdf8dd1c service glance] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:17:25.076509 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9615cf4-07b8-485f-8366-153afdf8dd1c service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:17:25.078142 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-a9615cf4-07b8-485f-8366-153afdf8dd1c service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:17:25.137949 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:17:25.156422 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:17:25.188680 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9615cf4-07b8-485f-8366-153afdf8dd1c service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:17:25.190258 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9615cf4-07b8-485f-8366-153afdf8dd1c service glance] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:17:25.190610 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da806741-1bf3-4fb6-ba0f-892e93403bed service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eface89e261840a2abb4206be26455c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '540a32ef88764e8c8d9201ae22f6ddaf', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:17:25.191260 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da806741-1bf3-4fb6-ba0f-892e93403bed service nova] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:17:25.191538 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da806741-1bf3-4fb6-ba0f-892e93403bed service nova] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:17:25.191808 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da806741-1bf3-4fb6-ba0f-892e93403bed service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:17:25.193475 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-da806741-1bf3-4fb6-ba0f-892e93403bed service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:17:25.210018 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 701/1395] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 12:17:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 12:17:25.288414 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da806741-1bf3-4fb6-ba0f-892e93403bed service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:17:25.290041 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da806741-1bf3-4fb6-ba0f-892e93403bed service nova] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:17:25.308079 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 695/1396] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 12:17:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 15 12:17:25.961988 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53beb50e-69a6-4869-9bed-b7e7735a758d None None] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:17:25.962480 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53beb50e-69a6-4869-9bed-b7e7735a758d None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:17:25.962589 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53beb50e-69a6-4869-9bed-b7e7735a758d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:17:26.031551 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.auth.core [None req-53beb50e-69a6-4869-9bed-b7e7735a758d None None] MFA Rules not processed for user `527f4dd6031546c785bc6360dacb60af`. Rule list: `[]` (Enabled: `True`). {{(pid=81449) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:17:26.032744 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-53beb50e-69a6-4869-9bed-b7e7735a758d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:17:26.118141 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 702/1397] 10.222.0.15 () {58 vars in 1009 bytes} [Thu Sep 15 12:17:25 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:17:26.161740 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27295df-3efb-4db7-939c-25547d612125 None None] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:17:26.162026 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27295df-3efb-4db7-939c-25547d612125 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:17:26.162300 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27295df-3efb-4db7-939c-25547d612125 None None] PATH_INFO: `/` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:17:26.163687 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 696/1398] 10.222.0.15 () {54 vars in 943 bytes} [Thu Sep 15 12:17:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:17:26.170178 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979fd9b8-47ae-40a2-8f38-cba420416564 None None] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:17:26.170479 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979fd9b8-47ae-40a2-8f38-cba420416564 None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:17:26.170752 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979fd9b8-47ae-40a2-8f38-cba420416564 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:17:26.230770 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:17:26.252587 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.auth.core [None req-979fd9b8-47ae-40a2-8f38-cba420416564 None None] MFA Rules not processed for user `efa0d3dc5b7540d59fdd59bbf64a04b6`. Rule list: `[]` (Enabled: `True`). {{(pid=81449) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:17:26.253852 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-979fd9b8-47ae-40a2-8f38-cba420416564 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:17:26.360164 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 703/1399] 10.222.0.15 () {58 vars in 1040 bytes} [Thu Sep 15 12:17:26 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:17:26.369444 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27295df-3efb-4db7-939c-25547d612125 None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:17:26.371014 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-d27295df-3efb-4db7-939c-25547d612125 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:17:26.446731 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:17:26.505877 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27295df-3efb-4db7-939c-25547d612125 None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:17:26.611493 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d5dc6d-4d1a-45f5-82f2-e54cd72486a0 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa0d3dc5b7540d59fdd59bbf64a04b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '540a32ef88764e8c8d9201ae22f6ddaf', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:17:26.612133 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d5dc6d-4d1a-45f5-82f2-e54cd72486a0 service swift] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:17:26.612397 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d5dc6d-4d1a-45f5-82f2-e54cd72486a0 service swift] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:17:26.612657 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d5dc6d-4d1a-45f5-82f2-e54cd72486a0 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:17:26.614318 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-81d5dc6d-4d1a-45f5-82f2-e54cd72486a0 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:17:26.679646 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:17:26.703390 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d5dc6d-4d1a-45f5-82f2-e54cd72486a0 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:17:26.704901 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d5dc6d-4d1a-45f5-82f2-e54cd72486a0 service swift] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:17:26.713308 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 697/1400] 10.222.0.15 () {60 vars in 1384 bytes} [Thu Sep 15 12:17:26 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 15 12:17:27.541921 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa1d8cc-206c-4c80-a684-f7f80cd14d2a None None] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:17:27.542453 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa1d8cc-206c-4c80-a684-f7f80cd14d2a None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:17:27.542453 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa1d8cc-206c-4c80-a684-f7f80cd14d2a None None] PATH_INFO: `/` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:17:27.543883 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 704/1401] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 12:17:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:17:27.549208 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bc8e6b-eb15-48b7-859a-28be4df6c8a0 None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:17:27.549657 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bc8e6b-eb15-48b7-859a-28be4df6c8a0 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:17:27.549844 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bc8e6b-eb15-48b7-859a-28be4df6c8a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:17:27.626465 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-19bc8e6b-eb15-48b7-859a-28be4df6c8a0 None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:17:27.628586 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-19bc8e6b-eb15-48b7-859a-28be4df6c8a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:17:27.746249 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 698/1402] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 12:17:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:17:28.056410 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa1d8cc-206c-4c80-a684-f7f80cd14d2a None None] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:17:28.059040 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-ffa1d8cc-206c-4c80-a684-f7f80cd14d2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:17:28.197716 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:17:28.252352 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa1d8cc-206c-4c80-a684-f7f80cd14d2a None None] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:17:28.375094 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1051654d-09b6-43bb-92c3-a1f7e9e6edfe service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eface89e261840a2abb4206be26455c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '540a32ef88764e8c8d9201ae22f6ddaf', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:17:28.375900 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1051654d-09b6-43bb-92c3-a1f7e9e6edfe service nova] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:17:28.376214 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1051654d-09b6-43bb-92c3-a1f7e9e6edfe service nova] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:17:28.376482 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1051654d-09b6-43bb-92c3-a1f7e9e6edfe service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:17:28.378148 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-1051654d-09b6-43bb-92c3-a1f7e9e6edfe service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:17:28.463131 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:17:28.487594 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1051654d-09b6-43bb-92c3-a1f7e9e6edfe service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:17:28.489235 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1051654d-09b6-43bb-92c3-a1f7e9e6edfe service nova] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:17:28.508097 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 705/1403] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 12:17:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 452 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 12:17:30.748628 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa91d05f-b517-4060-bf73-df0f38354bd2 None None] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:17:30.749869 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa91d05f-b517-4060-bf73-df0f38354bd2 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:17:30.750372 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa91d05f-b517-4060-bf73-df0f38354bd2 None None] PATH_INFO: `/` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:17:30.752177 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 699/1404] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 12:17:30 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:17:30.757344 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798b3251-df58-4766-bf2b-a233dbb8277c None None] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:17:30.757789 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798b3251-df58-4766-bf2b-a233dbb8277c None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:17:30.757920 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798b3251-df58-4766-bf2b-a233dbb8277c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:17:30.834741 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.auth.core [None req-798b3251-df58-4766-bf2b-a233dbb8277c None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81449) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:17:30.835960 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-798b3251-df58-4766-bf2b-a233dbb8277c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:17:30.941672 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 706/1405] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 12:17:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:17:31.235407 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa91d05f-b517-4060-bf73-df0f38354bd2 None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:17:31.236987 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-aa91d05f-b517-4060-bf73-df0f38354bd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:17:31.300910 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:17:31.391190 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa91d05f-b517-4060-bf73-df0f38354bd2 None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:17:31.519684 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbbd5bd6-9665-48f4-863b-d9b532994574 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eface89e261840a2abb4206be26455c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '540a32ef88764e8c8d9201ae22f6ddaf', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:17:31.521229 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbd5bd6-9665-48f4-863b-d9b532994574 service nova] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:17:31.521529 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbd5bd6-9665-48f4-863b-d9b532994574 service nova] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:17:31.521799 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbd5bd6-9665-48f4-863b-d9b532994574 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:17:31.523430 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-dbbd5bd6-9665-48f4-863b-d9b532994574 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:17:31.537801 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:17:31.620856 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbbd5bd6-9665-48f4-863b-d9b532994574 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:17:31.623032 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbbd5bd6-9665-48f4-863b-d9b532994574 service nova] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:17:31.644272 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 700/1406] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 12:17:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 409 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 12:17:33.878670 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecfc07d-ee27-454d-b723-33ad40827166 None None] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:17:33.879248 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecfc07d-ee27-454d-b723-33ad40827166 None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:17:33.879248 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fecfc07d-ee27-454d-b723-33ad40827166 None None] PATH_INFO: `/` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:17:33.881164 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 707/1407] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 12:17:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:17:33.886366 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9423c319-a1b3-4b1c-b9a6-4fbfb01ff1bf None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:17:33.886840 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9423c319-a1b3-4b1c-b9a6-4fbfb01ff1bf None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:17:33.886935 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9423c319-a1b3-4b1c-b9a6-4fbfb01ff1bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:17:33.969391 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-9423c319-a1b3-4b1c-b9a6-4fbfb01ff1bf None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:17:33.970758 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-9423c319-a1b3-4b1c-b9a6-4fbfb01ff1bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:17:34.067813 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:17:34.079755 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 701/1408] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 12:17:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 12:17:34.373120 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fecfc07d-ee27-454d-b723-33ad40827166 None None] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:17:34.374653 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-fecfc07d-ee27-454d-b723-33ad40827166 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:17:34.438513 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:17:34.547833 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fecfc07d-ee27-454d-b723-33ad40827166 None None] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:17:34.681990 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd655e4-7d3e-4c9e-a046-d5ea5c659aa0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eface89e261840a2abb4206be26455c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '540a32ef88764e8c8d9201ae22f6ddaf', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:17:34.682604 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd655e4-7d3e-4c9e-a046-d5ea5c659aa0 service nova] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:17:34.682881 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd655e4-7d3e-4c9e-a046-d5ea5c659aa0 service nova] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:17:34.683151 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd655e4-7d3e-4c9e-a046-d5ea5c659aa0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:17:34.684893 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-ddd655e4-7d3e-4c9e-a046-d5ea5c659aa0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:17:34.692528 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:17:34.788014 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd655e4-7d3e-4c9e-a046-d5ea5c659aa0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:17:34.789755 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd655e4-7d3e-4c9e-a046-d5ea5c659aa0 service nova] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:17:34.808839 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 708/1409] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 12:17:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 437 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 12:17:44.078172 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=4 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:17:44.806240 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=42 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:17:49.420445 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e3914b-f8a1-44bc-9ecb-7eb3f770c09c None None] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:17:49.421177 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e3914b-f8a1-44bc-9ecb-7eb3f770c09c None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:17:49.421177 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e3914b-f8a1-44bc-9ecb-7eb3f770c09c None None] PATH_INFO: `/` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:17:49.422367 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 702/1410] 10.222.0.15 () {54 vars in 918 bytes} [Thu Sep 15 12:17:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:17:49.427436 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8902c2-2c0a-4e15-b09b-36aa0bf9b878 None None] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:17:49.427859 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8902c2-2c0a-4e15-b09b-36aa0bf9b878 None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:17:49.427970 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8902c2-2c0a-4e15-b09b-36aa0bf9b878 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:17:49.504363 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.auth.core [None req-ed8902c2-2c0a-4e15-b09b-36aa0bf9b878 None None] MFA Rules not processed for user `51f64a058bd5440cb5268f228aa6e392`. Rule list: `[]` (Enabled: `True`). {{(pid=81449) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:17:49.505597 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-ed8902c2-2c0a-4e15-b09b-36aa0bf9b878 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:17:49.612355 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 709/1411] 10.222.0.15 () {58 vars in 1015 bytes} [Thu Sep 15 12:17:49 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:17:49.627551 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6887ec3b-c88f-4938-af6c-f7aa6f512f2e None None] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:17:49.627832 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6887ec3b-c88f-4938-af6c-f7aa6f512f2e None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:17:49.628103 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6887ec3b-c88f-4938-af6c-f7aa6f512f2e None None] PATH_INFO: `/` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:17:49.629452 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 703/1412] 10.222.0.15 () {54 vars in 918 bytes} [Thu Sep 15 12:17:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:17:49.634109 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabd946b-9c95-449d-b6f3-bfe0a065a279 None None] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:17:49.634389 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabd946b-9c95-449d-b6f3-bfe0a065a279 None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:17:49.634656 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabd946b-9c95-449d-b6f3-bfe0a065a279 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:17:49.707170 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:17:49.710326 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.auth.core [None req-dabd946b-9c95-449d-b6f3-bfe0a065a279 None None] MFA Rules not processed for user `eface89e261840a2abb4206be26455c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81449) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:17:49.711942 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-dabd946b-9c95-449d-b6f3-bfe0a065a279 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:17:49.825340 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 710/1413] 10.222.0.15 () {58 vars in 1015 bytes} [Thu Sep 15 12:17:49 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:17:52.371275 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1dde3e-4158-4191-9fb4-2fda0c07ba58 None None] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:17:52.371873 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1dde3e-4158-4191-9fb4-2fda0c07ba58 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:17:52.371873 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1dde3e-4158-4191-9fb4-2fda0c07ba58 None None] PATH_INFO: `/` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:17:52.373176 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 704/1414] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 15 12:17:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:17:52.378488 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af69cc1-d598-41eb-99cd-3bffdf82ba95 None None] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:17:52.378905 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af69cc1-d598-41eb-99cd-3bffdf82ba95 None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:17:52.379038 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af69cc1-d598-41eb-99cd-3bffdf82ba95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:17:52.443926 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.auth.core [None req-3af69cc1-d598-41eb-99cd-3bffdf82ba95 None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81449) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:17:52.445142 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-3af69cc1-d598-41eb-99cd-3bffdf82ba95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:17:52.501193 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:17:52.543091 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 711/1415] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 15 12:17:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:17:52.844091 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1dde3e-4158-4191-9fb4-2fda0c07ba58 None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:17:52.845753 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-df1dde3e-4158-4191-9fb4-2fda0c07ba58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:17:52.992925 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1dde3e-4158-4191-9fb4-2fda0c07ba58 None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:17:53.065239 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:17:53.107368 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371631ac-bdc8-4dd8-92c0-0c58a649a61a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eface89e261840a2abb4206be26455c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '540a32ef88764e8c8d9201ae22f6ddaf', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:17:53.107901 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371631ac-bdc8-4dd8-92c0-0c58a649a61a service nova] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:17:53.108381 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371631ac-bdc8-4dd8-92c0-0c58a649a61a service nova] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:17:53.108736 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371631ac-bdc8-4dd8-92c0-0c58a649a61a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:17:53.110489 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-371631ac-bdc8-4dd8-92c0-0c58a649a61a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:17:53.199966 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371631ac-bdc8-4dd8-92c0-0c58a649a61a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:17:53.201641 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371631ac-bdc8-4dd8-92c0-0c58a649a61a service nova] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:17:53.219490 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 705/1416] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 12:17:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 12:17:54.346173 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882aa68f-ecb8-4aeb-9865-61cd1e8ffa85 None None] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:17:54.346711 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882aa68f-ecb8-4aeb-9865-61cd1e8ffa85 None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:17:54.346711 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882aa68f-ecb8-4aeb-9865-61cd1e8ffa85 None None] PATH_INFO: `/` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:17:54.348052 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 712/1417] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 15 12:17:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:17:54.354043 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440abeb1-ee0e-4cab-b1b8-8d6e74ac3667 None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:17:54.354483 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440abeb1-ee0e-4cab-b1b8-8d6e74ac3667 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:17:54.354587 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440abeb1-ee0e-4cab-b1b8-8d6e74ac3667 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:17:54.429389 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-440abeb1-ee0e-4cab-b1b8-8d6e74ac3667 None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:17:54.430581 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-440abeb1-ee0e-4cab-b1b8-8d6e74ac3667 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:17:54.457902 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:17:54.526916 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 706/1418] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 15 12:17:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:17:54.678797 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882aa68f-ecb8-4aeb-9865-61cd1e8ffa85 None None] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:17:54.680226 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-882aa68f-ecb8-4aeb-9865-61cd1e8ffa85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:17:54.832056 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882aa68f-ecb8-4aeb-9865-61cd1e8ffa85 None None] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:17:54.871990 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:17:54.947674 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46bb71e6-30c5-4387-8baf-0ab77d100d03 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eface89e261840a2abb4206be26455c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '540a32ef88764e8c8d9201ae22f6ddaf', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:17:54.948312 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bb71e6-30c5-4387-8baf-0ab77d100d03 service nova] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:17:54.948594 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bb71e6-30c5-4387-8baf-0ab77d100d03 service nova] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:17:54.948905 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bb71e6-30c5-4387-8baf-0ab77d100d03 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:17:54.950524 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-46bb71e6-30c5-4387-8baf-0ab77d100d03 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:17:55.039634 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46bb71e6-30c5-4387-8baf-0ab77d100d03 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:17:55.041272 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46bb71e6-30c5-4387-8baf-0ab77d100d03 service nova] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:17:55.059615 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 713/1419] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 12:17:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 12:17:56.184666 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5f0e96-0048-4db9-b378-11c4aab454db None None] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:17:56.185802 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5f0e96-0048-4db9-b378-11c4aab454db None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:17:56.185902 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc5f0e96-0048-4db9-b378-11c4aab454db None None] PATH_INFO: `/` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:17:56.187248 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 707/1420] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 15 12:17:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 15 12:17:56.193697 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff852e4d-2881-4ca5-82b5-7cdd187270d8 None None] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:17:56.194134 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff852e4d-2881-4ca5-82b5-7cdd187270d8 None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:17:56.194230 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff852e4d-2881-4ca5-82b5-7cdd187270d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:17:56.248419 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:17:56.270179 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.auth.core [None req-ff852e4d-2881-4ca5-82b5-7cdd187270d8 None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81449) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:17:56.271379 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-ff852e4d-2881-4ca5-82b5-7cdd187270d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:17:56.365928 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 714/1421] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 15 12:17:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:17:56.516980 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5f0e96-0048-4db9-b378-11c4aab454db None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:17:56.518395 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-dc5f0e96-0048-4db9-b378-11c4aab454db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:17:56.661716 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc5f0e96-0048-4db9-b378-11c4aab454db None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:17:56.673864 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:17:56.773791 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce595ee4-4ddd-46ba-9f6c-4f434f8f310e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eface89e261840a2abb4206be26455c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '540a32ef88764e8c8d9201ae22f6ddaf', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:17:56.774311 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce595ee4-4ddd-46ba-9f6c-4f434f8f310e service nova] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:17:56.774586 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce595ee4-4ddd-46ba-9f6c-4f434f8f310e service nova] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:17:56.774851 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce595ee4-4ddd-46ba-9f6c-4f434f8f310e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:17:56.776418 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-ce595ee4-4ddd-46ba-9f6c-4f434f8f310e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:17:56.863596 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce595ee4-4ddd-46ba-9f6c-4f434f8f310e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:17:56.865199 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce595ee4-4ddd-46ba-9f6c-4f434f8f310e service nova] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:17:56.882976 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 708/1422] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 12:17:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 12:17:57.987281 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72bd307-ad48-4913-b92e-6b2066101d9f None None] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:17:57.987790 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72bd307-ad48-4913-b92e-6b2066101d9f None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:17:57.987898 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72bd307-ad48-4913-b92e-6b2066101d9f None None] PATH_INFO: `/` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:17:57.989233 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 715/1423] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 15 12:17:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:17:57.995510 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee9d25a-d835-4e19-a26c-e87aaa0c6428 None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:17:57.995949 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee9d25a-d835-4e19-a26c-e87aaa0c6428 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:17:57.996049 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee9d25a-d835-4e19-a26c-e87aaa0c6428 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:17:58.015185 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:17:58.188368 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-5ee9d25a-d835-4e19-a26c-e87aaa0c6428 None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:17:58.189709 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-5ee9d25a-d835-4e19-a26c-e87aaa0c6428 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:17:58.285835 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 709/1424] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 15 12:17:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 291 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:17:58.439336 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72bd307-ad48-4913-b92e-6b2066101d9f None None] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:17:58.440862 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-b72bd307-ad48-4913-b92e-6b2066101d9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:17:58.550156 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:17:58.585928 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72bd307-ad48-4913-b92e-6b2066101d9f None None] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:17:58.696706 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2514ff06-e0a7-4c64-86af-44d43cf7854f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eface89e261840a2abb4206be26455c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '540a32ef88764e8c8d9201ae22f6ddaf', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:17:58.697334 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2514ff06-e0a7-4c64-86af-44d43cf7854f service nova] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:17:58.697608 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2514ff06-e0a7-4c64-86af-44d43cf7854f service nova] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:17:58.697874 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2514ff06-e0a7-4c64-86af-44d43cf7854f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:17:58.699441 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-2514ff06-e0a7-4c64-86af-44d43cf7854f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:17:58.776542 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:17:58.791563 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2514ff06-e0a7-4c64-86af-44d43cf7854f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:17:58.793222 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2514ff06-e0a7-4c64-86af-44d43cf7854f service nova] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:17:58.811753 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 716/1425] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 12:17:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 12:17:59.932704 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df89ffab-935a-416e-b636-711cbaca8c76 None None] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:17:59.933320 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df89ffab-935a-416e-b636-711cbaca8c76 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:17:59.933320 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df89ffab-935a-416e-b636-711cbaca8c76 None None] PATH_INFO: `/` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:17:59.934620 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 710/1426] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 15 12:17:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:17:59.940993 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9546e0-1e8b-4bf6-aa19-beb332e33ca2 None None] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:17:59.941414 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9546e0-1e8b-4bf6-aa19-beb332e33ca2 None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:17:59.941548 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9546e0-1e8b-4bf6-aa19-beb332e33ca2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:18:00.017676 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.auth.core [None req-3d9546e0-1e8b-4bf6-aa19-beb332e33ca2 None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81449) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:18:00.018875 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-3d9546e0-1e8b-4bf6-aa19-beb332e33ca2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:18:00.112890 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 717/1427] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 15 12:17:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:18:00.266284 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df89ffab-935a-416e-b636-711cbaca8c76 None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:18:00.267731 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-df89ffab-935a-416e-b636-711cbaca8c76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:18:00.349687 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:18:00.412069 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df89ffab-935a-416e-b636-711cbaca8c76 None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:18:00.529695 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fb38d3-d646-49a7-9506-1abff10aab95 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eface89e261840a2abb4206be26455c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '540a32ef88764e8c8d9201ae22f6ddaf', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:18:00.530193 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fb38d3-d646-49a7-9506-1abff10aab95 service nova] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:18:00.530467 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fb38d3-d646-49a7-9506-1abff10aab95 service nova] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:18:00.530733 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fb38d3-d646-49a7-9506-1abff10aab95 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:18:00.532315 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-43fb38d3-d646-49a7-9506-1abff10aab95 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:18:00.579514 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:18:00.620367 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43fb38d3-d646-49a7-9506-1abff10aab95 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:18:00.621957 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43fb38d3-d646-49a7-9506-1abff10aab95 service nova] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:18:00.640144 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 711/1428] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 12:18:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 12:18:01.690997 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148467cc-1f3d-42b6-9f32-50e990ad39a3 None None] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:18:01.691478 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148467cc-1f3d-42b6-9f32-50e990ad39a3 None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:18:01.691585 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148467cc-1f3d-42b6-9f32-50e990ad39a3 None None] PATH_INFO: `/` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:18:01.692928 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 718/1429] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 15 12:18:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:18:01.699069 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1cbfbb-6113-4fbc-899c-2bb64e42ec54 None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:18:01.699518 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1cbfbb-6113-4fbc-899c-2bb64e42ec54 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:18:01.699637 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1cbfbb-6113-4fbc-899c-2bb64e42ec54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:18:01.779256 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-7c1cbfbb-6113-4fbc-899c-2bb64e42ec54 None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:18:01.780459 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-7c1cbfbb-6113-4fbc-899c-2bb64e42ec54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:18:01.875889 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 712/1430] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 15 12:18:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:18:02.026102 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148467cc-1f3d-42b6-9f32-50e990ad39a3 None None] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:18:02.027624 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-148467cc-1f3d-42b6-9f32-50e990ad39a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:18:02.077239 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:18:02.174598 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148467cc-1f3d-42b6-9f32-50e990ad39a3 None None] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:18:02.287816 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d761d533-20a3-4a76-9f82-4abc34b7a929 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eface89e261840a2abb4206be26455c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '540a32ef88764e8c8d9201ae22f6ddaf', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:18:02.288333 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d761d533-20a3-4a76-9f82-4abc34b7a929 service nova] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:18:02.288607 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d761d533-20a3-4a76-9f82-4abc34b7a929 service nova] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:18:02.288905 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d761d533-20a3-4a76-9f82-4abc34b7a929 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:18:02.290474 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-d761d533-20a3-4a76-9f82-4abc34b7a929 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:18:02.297720 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:18:02.378550 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d761d533-20a3-4a76-9f82-4abc34b7a929 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:18:02.380113 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d761d533-20a3-4a76-9f82-4abc34b7a929 service nova] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:18:02.397666 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 719/1431] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 12:18:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 12:18:03.442071 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2980c8-e571-4e8a-b518-8f98682e0bbe None None] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:18:03.442622 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2980c8-e571-4e8a-b518-8f98682e0bbe None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:18:03.442622 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2980c8-e571-4e8a-b518-8f98682e0bbe None None] PATH_INFO: `/` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:18:03.443981 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 713/1432] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 15 12:18:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:18:03.449974 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b31d6b3-ab4b-48f4-b837-ae7b26ca1154 None None] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:18:03.450422 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b31d6b3-ab4b-48f4-b837-ae7b26ca1154 None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:18:03.451122 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b31d6b3-ab4b-48f4-b837-ae7b26ca1154 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:18:03.525651 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.auth.core [None req-8b31d6b3-ab4b-48f4-b837-ae7b26ca1154 None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81449) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:18:03.526839 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-8b31d6b3-ab4b-48f4-b837-ae7b26ca1154 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:18:03.592219 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:18:03.623250 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 720/1433] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 15 12:18:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:18:03.773365 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2980c8-e571-4e8a-b518-8f98682e0bbe None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:18:03.774929 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-fa2980c8-e571-4e8a-b518-8f98682e0bbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:18:03.792078 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:18:03.916465 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2980c8-e571-4e8a-b518-8f98682e0bbe None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:18:04.002256 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:18:04.027247 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf411bbc-85d9-4bc9-8649-f402d7f34780 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eface89e261840a2abb4206be26455c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '540a32ef88764e8c8d9201ae22f6ddaf', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:18:04.027881 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf411bbc-85d9-4bc9-8649-f402d7f34780 service nova] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:18:04.028158 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf411bbc-85d9-4bc9-8649-f402d7f34780 service nova] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:18:04.028449 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf411bbc-85d9-4bc9-8649-f402d7f34780 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:18:04.030571 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-cf411bbc-85d9-4bc9-8649-f402d7f34780 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:18:04.118806 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf411bbc-85d9-4bc9-8649-f402d7f34780 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:18:04.120364 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf411bbc-85d9-4bc9-8649-f402d7f34780 service nova] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:18:04.137731 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 714/1434] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 12:18:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 12:18:05.181789 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea6752a-bdca-49a7-be3a-5f9a579f5375 None None] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:18:05.182350 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea6752a-bdca-49a7-be3a-5f9a579f5375 None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:18:05.182350 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea6752a-bdca-49a7-be3a-5f9a579f5375 None None] PATH_INFO: `/` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:18:05.183665 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 721/1435] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 15 12:18:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:18:05.189834 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c7019f-326b-4f46-952f-3a74de4657aa None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:18:05.190352 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c7019f-326b-4f46-952f-3a74de4657aa None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:18:05.190352 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c7019f-326b-4f46-952f-3a74de4657aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:18:05.265363 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-36c7019f-326b-4f46-952f-3a74de4657aa None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:18:05.266477 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-36c7019f-326b-4f46-952f-3a74de4657aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:18:05.306636 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:18:05.362137 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 715/1436] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 15 12:18:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 12:18:05.510297 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea6752a-bdca-49a7-be3a-5f9a579f5375 None None] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:18:05.511804 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-dea6752a-bdca-49a7-be3a-5f9a579f5375 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:18:05.655941 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea6752a-bdca-49a7-be3a-5f9a579f5375 None None] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:18:05.708905 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:18:05.768351 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64445fe-9020-4484-a951-80e1a86292cb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eface89e261840a2abb4206be26455c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '540a32ef88764e8c8d9201ae22f6ddaf', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:18:05.768994 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64445fe-9020-4484-a951-80e1a86292cb service nova] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:18:05.769272 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64445fe-9020-4484-a951-80e1a86292cb service nova] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:18:05.769540 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64445fe-9020-4484-a951-80e1a86292cb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:18:05.771139 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-d64445fe-9020-4484-a951-80e1a86292cb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:18:05.859880 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64445fe-9020-4484-a951-80e1a86292cb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:18:05.861504 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64445fe-9020-4484-a951-80e1a86292cb service nova] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:18:05.879462 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 722/1437] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 12:18:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 12:18:06.931134 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0423e9f6-1e1c-4949-ac0d-15add4b4e6f3 None None] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:18:06.931713 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0423e9f6-1e1c-4949-ac0d-15add4b4e6f3 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:18:06.931713 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0423e9f6-1e1c-4949-ac0d-15add4b4e6f3 None None] PATH_INFO: `/` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:18:06.933055 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 716/1438] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 15 12:18:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:18:06.939650 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7668f991-4fb0-481c-a2c5-33b924c73610 None None] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:18:06.940110 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7668f991-4fb0-481c-a2c5-33b924c73610 None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:18:06.940216 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7668f991-4fb0-481c-a2c5-33b924c73610 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:18:07.018766 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.auth.core [None req-7668f991-4fb0-481c-a2c5-33b924c73610 None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81449) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:18:07.019975 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-7668f991-4fb0-481c-a2c5-33b924c73610 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:18:07.022046 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:18:07.117920 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 723/1439] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 15 12:18:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:18:07.267428 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0423e9f6-1e1c-4949-ac0d-15add4b4e6f3 None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:18:07.269004 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-0423e9f6-1e1c-4949-ac0d-15add4b4e6f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:18:07.411013 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0423e9f6-1e1c-4949-ac0d-15add4b4e6f3 None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:18:07.437970 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:18:07.521995 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290cb73e-2e9e-430a-b134-e02116efeb03 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eface89e261840a2abb4206be26455c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '540a32ef88764e8c8d9201ae22f6ddaf', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:18:07.522533 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290cb73e-2e9e-430a-b134-e02116efeb03 service nova] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:18:07.522813 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290cb73e-2e9e-430a-b134-e02116efeb03 service nova] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:18:07.523083 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290cb73e-2e9e-430a-b134-e02116efeb03 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:18:07.524679 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-290cb73e-2e9e-430a-b134-e02116efeb03 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:18:07.613103 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-290cb73e-2e9e-430a-b134-e02116efeb03 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:18:07.614672 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-290cb73e-2e9e-430a-b134-e02116efeb03 service nova] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:18:07.632707 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 717/1440] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 12:18:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 12:18:09.701232 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668cec04-d74a-4ce9-a2e5-5be1754b98c4 None None] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:18:09.701775 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668cec04-d74a-4ce9-a2e5-5be1754b98c4 None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:18:09.701775 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668cec04-d74a-4ce9-a2e5-5be1754b98c4 None None] PATH_INFO: `/` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:18:09.703151 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 724/1441] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 15 12:18:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:18:09.708507 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68af18db-98d2-4e76-8d93-90d7ae0caaa6 None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:18:09.708972 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68af18db-98d2-4e76-8d93-90d7ae0caaa6 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:18:09.709150 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68af18db-98d2-4e76-8d93-90d7ae0caaa6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:18:09.749911 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:18:09.776047 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-68af18db-98d2-4e76-8d93-90d7ae0caaa6 None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:18:09.777275 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-68af18db-98d2-4e76-8d93-90d7ae0caaa6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:18:09.872087 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 718/1442] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 15 12:18:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:18:10.156369 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668cec04-d74a-4ce9-a2e5-5be1754b98c4 None None] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:18:10.158658 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-668cec04-d74a-4ce9-a2e5-5be1754b98c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:18:10.268167 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-668cec04-d74a-4ce9-a2e5-5be1754b98c4 None None] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:18:10.301454 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:18:10.343025 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313d464d-15bb-4d83-98bb-bf35fc7bf410 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfbb4d6809064f9cb9e06626ed7d6814', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '540a32ef88764e8c8d9201ae22f6ddaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:18:10.343631 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313d464d-15bb-4d83-98bb-bf35fc7bf410 service neutron] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:18:10.343906 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313d464d-15bb-4d83-98bb-bf35fc7bf410 service neutron] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:18:10.344425 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313d464d-15bb-4d83-98bb-bf35fc7bf410 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:18:10.346053 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-313d464d-15bb-4d83-98bb-bf35fc7bf410 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:18:10.453453 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313d464d-15bb-4d83-98bb-bf35fc7bf410 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:18:10.454880 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313d464d-15bb-4d83-98bb-bf35fc7bf410 service neutron] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:18:10.473035 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 725/1443] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 12:18:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 12:18:12.754541 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735c760c-8a63-4087-be87-dcb216ad6b36 None None] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:18:12.755112 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735c760c-8a63-4087-be87-dcb216ad6b36 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:18:12.755112 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735c760c-8a63-4087-be87-dcb216ad6b36 None None] PATH_INFO: `/` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:18:12.756449 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 719/1444] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 15 12:18:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:18:12.761904 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e37a75-0a7b-41bc-9ba1-f715fe0173bd None None] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:18:12.762315 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e37a75-0a7b-41bc-9ba1-f715fe0173bd None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:18:12.762455 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e37a75-0a7b-41bc-9ba1-f715fe0173bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:18:12.831804 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.auth.core [None req-99e37a75-0a7b-41bc-9ba1-f715fe0173bd None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81449) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:18:12.833052 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-99e37a75-0a7b-41bc-9ba1-f715fe0173bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:18:12.847739 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:18:12.934661 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 726/1445] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 15 12:18:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:18:13.220325 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735c760c-8a63-4087-be87-dcb216ad6b36 None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:18:13.221921 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-735c760c-8a63-4087-be87-dcb216ad6b36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:18:13.322516 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735c760c-8a63-4087-be87-dcb216ad6b36 None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:18:13.335406 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:18:13.395875 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6687442-0ced-4092-b308-59f41a6c4bb6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfbb4d6809064f9cb9e06626ed7d6814', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '540a32ef88764e8c8d9201ae22f6ddaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:18:13.396479 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6687442-0ced-4092-b308-59f41a6c4bb6 service neutron] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:18:13.396770 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6687442-0ced-4092-b308-59f41a6c4bb6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:18:13.397043 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6687442-0ced-4092-b308-59f41a6c4bb6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:18:13.398625 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-e6687442-0ced-4092-b308-59f41a6c4bb6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:18:13.504612 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6687442-0ced-4092-b308-59f41a6c4bb6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:18:13.505979 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6687442-0ced-4092-b308-59f41a6c4bb6 service neutron] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:18:13.525240 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 720/1446] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 12:18:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 12:18:15.970085 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83646ab-193e-45e6-8440-73b639bf48d2 None None] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:18:15.970636 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83646ab-193e-45e6-8440-73b639bf48d2 None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:18:15.970636 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83646ab-193e-45e6-8440-73b639bf48d2 None None] PATH_INFO: `/` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:18:15.973443 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 727/1447] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 15 12:18:15 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:18:15.979199 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d6a6b0-998d-44f2-92cf-2b50fe919bde None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:18:15.979649 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d6a6b0-998d-44f2-92cf-2b50fe919bde None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:18:15.979743 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d6a6b0-998d-44f2-92cf-2b50fe919bde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:18:16.028096 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:18:16.046904 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-b7d6a6b0-998d-44f2-92cf-2b50fe919bde None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:18:16.048098 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-b7d6a6b0-998d-44f2-92cf-2b50fe919bde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:18:16.142234 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 721/1448] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 15 12:18:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:18:16.434809 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83646ab-193e-45e6-8440-73b639bf48d2 None None] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:18:16.436343 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-a83646ab-193e-45e6-8440-73b639bf48d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:18:16.538314 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83646ab-193e-45e6-8440-73b639bf48d2 None None] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:18:16.577231 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:18:16.611079 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519ce578-73b8-4393-a66c-85275d189636 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfbb4d6809064f9cb9e06626ed7d6814', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '540a32ef88764e8c8d9201ae22f6ddaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:18:16.611688 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519ce578-73b8-4393-a66c-85275d189636 service neutron] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:18:16.611971 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519ce578-73b8-4393-a66c-85275d189636 service neutron] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:18:16.612241 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519ce578-73b8-4393-a66c-85275d189636 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:18:16.613902 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-519ce578-73b8-4393-a66c-85275d189636 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:18:16.726913 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519ce578-73b8-4393-a66c-85275d189636 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:18:16.728348 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519ce578-73b8-4393-a66c-85275d189636 service neutron] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:18:16.746254 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 728/1449] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 12:18:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 12:18:18.988011 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6822ee83-520f-496c-ac37-a80c8ab3ad7c None None] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:18:18.988459 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6822ee83-520f-496c-ac37-a80c8ab3ad7c None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:18:18.988587 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6822ee83-520f-496c-ac37-a80c8ab3ad7c None None] PATH_INFO: `/` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:18:18.990268 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 722/1450] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 15 12:18:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:18:18.995446 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd96d8f-5d9d-4392-acb5-705603648c65 None None] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:18:18.995933 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd96d8f-5d9d-4392-acb5-705603648c65 None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:18:18.995933 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd96d8f-5d9d-4392-acb5-705603648c65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:18:19.064610 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.auth.core [None req-4fd96d8f-5d9d-4392-acb5-705603648c65 None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81449) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:18:19.065837 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-4fd96d8f-5d9d-4392-acb5-705603648c65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:18:19.086292 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:18:19.162670 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 729/1451] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 15 12:18:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:18:19.447871 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6822ee83-520f-496c-ac37-a80c8ab3ad7c None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:18:19.449437 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-6822ee83-520f-496c-ac37-a80c8ab3ad7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:18:19.551869 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6822ee83-520f-496c-ac37-a80c8ab3ad7c None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:18:19.571887 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:18:19.628738 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786930f3-fdef-49c4-a223-19fdaf6fe309 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfbb4d6809064f9cb9e06626ed7d6814', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '540a32ef88764e8c8d9201ae22f6ddaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:18:19.629364 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786930f3-fdef-49c4-a223-19fdaf6fe309 service neutron] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:18:19.629643 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786930f3-fdef-49c4-a223-19fdaf6fe309 service neutron] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:18:19.629908 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786930f3-fdef-49c4-a223-19fdaf6fe309 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:18:19.631485 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-786930f3-fdef-49c4-a223-19fdaf6fe309 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:18:19.738483 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786930f3-fdef-49c4-a223-19fdaf6fe309 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:18:19.739901 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786930f3-fdef-49c4-a223-19fdaf6fe309 service neutron] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:18:19.757406 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 723/1452] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 12:18:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 12:18:21.959562 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8bc423-da79-4ff7-8e95-3223274e4a43 None None] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:18:21.960115 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8bc423-da79-4ff7-8e95-3223274e4a43 None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:18:21.960115 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8bc423-da79-4ff7-8e95-3223274e4a43 None None] PATH_INFO: `/` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:18:21.961472 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 730/1453] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 15 12:18:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:18:21.966853 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5b8f93-a890-4ff1-8aa7-a79ece7673ee None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:18:21.967266 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5b8f93-a890-4ff1-8aa7-a79ece7673ee None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:18:21.967400 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5b8f93-a890-4ff1-8aa7-a79ece7673ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:18:22.031928 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:18:22.034545 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-fd5b8f93-a890-4ff1-8aa7-a79ece7673ee None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:18:22.036360 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-fd5b8f93-a890-4ff1-8aa7-a79ece7673ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:18:22.132290 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 724/1454] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 15 12:18:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:18:22.419173 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8bc423-da79-4ff7-8e95-3223274e4a43 None None] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:18:22.421354 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-7a8bc423-da79-4ff7-8e95-3223274e4a43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:18:22.522900 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8bc423-da79-4ff7-8e95-3223274e4a43 None None] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:18:22.571129 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:18:22.596493 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da856f6-fe3a-4e05-ab1f-431a7d8ea83d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfbb4d6809064f9cb9e06626ed7d6814', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '540a32ef88764e8c8d9201ae22f6ddaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:18:22.597122 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da856f6-fe3a-4e05-ab1f-431a7d8ea83d service neutron] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:18:22.597398 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da856f6-fe3a-4e05-ab1f-431a7d8ea83d service neutron] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:18:22.597662 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da856f6-fe3a-4e05-ab1f-431a7d8ea83d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:18:22.599217 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-5da856f6-fe3a-4e05-ab1f-431a7d8ea83d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:18:22.704733 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da856f6-fe3a-4e05-ab1f-431a7d8ea83d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:18:22.706185 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da856f6-fe3a-4e05-ab1f-431a7d8ea83d service neutron] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:18:22.724579 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 731/1455] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 12:18:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 12:18:24.938231 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbcc6ee-e889-487a-aa0f-8d8518e57a97 None None] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:18:24.938800 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbcc6ee-e889-487a-aa0f-8d8518e57a97 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:18:24.938800 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbcc6ee-e889-487a-aa0f-8d8518e57a97 None None] PATH_INFO: `/` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:18:24.940158 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 725/1456] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 15 12:18:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:18:24.945797 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244f1590-ec6e-402a-bff2-ba2640a5df31 None None] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:18:24.946231 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244f1590-ec6e-402a-bff2-ba2640a5df31 None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:18:24.946328 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244f1590-ec6e-402a-bff2-ba2640a5df31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:18:25.011175 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.auth.core [None req-244f1590-ec6e-402a-bff2-ba2640a5df31 None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81449) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:18:25.012372 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-244f1590-ec6e-402a-bff2-ba2640a5df31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:18:25.040860 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:18:25.110018 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 732/1457] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 15 12:18:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:18:25.396471 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbcc6ee-e889-487a-aa0f-8d8518e57a97 None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:18:25.398076 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-dcbcc6ee-e889-487a-aa0f-8d8518e57a97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:18:25.499900 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbcc6ee-e889-487a-aa0f-8d8518e57a97 None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:18:25.527858 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:18:25.572548 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b58607-565c-4301-8a59-157b0e4a25c4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfbb4d6809064f9cb9e06626ed7d6814', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '540a32ef88764e8c8d9201ae22f6ddaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:18:25.573182 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b58607-565c-4301-8a59-157b0e4a25c4 service neutron] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:18:25.573464 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b58607-565c-4301-8a59-157b0e4a25c4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:18:25.573732 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b58607-565c-4301-8a59-157b0e4a25c4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:18:25.575306 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-63b58607-565c-4301-8a59-157b0e4a25c4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:18:25.688775 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b58607-565c-4301-8a59-157b0e4a25c4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:18:25.690228 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b58607-565c-4301-8a59-157b0e4a25c4 service neutron] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:18:25.707661 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 726/1458] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 12:18:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 12:18:28.148556 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d8992b-ea8e-4ce9-8460-c8e34f3b5e74 None None] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:18:28.149166 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d8992b-ea8e-4ce9-8460-c8e34f3b5e74 None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:18:28.149166 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d8992b-ea8e-4ce9-8460-c8e34f3b5e74 None None] PATH_INFO: `/` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:18:28.150530 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 733/1459] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 15 12:18:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:18:28.156008 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee09198-66f1-41a5-b0c4-fdf4ea80c241 None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:18:28.156522 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee09198-66f1-41a5-b0c4-fdf4ea80c241 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:18:28.156629 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee09198-66f1-41a5-b0c4-fdf4ea80c241 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:18:28.223271 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-0ee09198-66f1-41a5-b0c4-fdf4ea80c241 None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:18:28.224467 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-0ee09198-66f1-41a5-b0c4-fdf4ea80c241 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:18:28.232805 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:18:28.319234 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 727/1460] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 15 12:18:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:18:28.601698 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d8992b-ea8e-4ce9-8460-c8e34f3b5e74 None None] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:18:28.603495 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-49d8992b-ea8e-4ce9-8460-c8e34f3b5e74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:18:28.711448 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d8992b-ea8e-4ce9-8460-c8e34f3b5e74 None None] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:18:28.767213 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:18:28.785019 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fd844d-bae9-4483-93cb-7b3fc474fb00 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfbb4d6809064f9cb9e06626ed7d6814', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '540a32ef88764e8c8d9201ae22f6ddaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:18:28.785623 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fd844d-bae9-4483-93cb-7b3fc474fb00 service neutron] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:18:28.785902 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fd844d-bae9-4483-93cb-7b3fc474fb00 service neutron] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:18:28.786178 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fd844d-bae9-4483-93cb-7b3fc474fb00 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:18:28.787784 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-81fd844d-bae9-4483-93cb-7b3fc474fb00 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:18:28.900045 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81fd844d-bae9-4483-93cb-7b3fc474fb00 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:18:28.901574 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81fd844d-bae9-4483-93cb-7b3fc474fb00 service neutron] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:18:28.920360 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 734/1461] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 12:18:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 12:18:31.258037 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f3f12a-34aa-437d-bddf-6ab8fbc76642 None None] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:18:31.258594 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f3f12a-34aa-437d-bddf-6ab8fbc76642 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:18:31.258594 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f3f12a-34aa-437d-bddf-6ab8fbc76642 None None] PATH_INFO: `/` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:18:31.259873 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 728/1462] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 15 12:18:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:18:31.266041 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7799a2-207d-43e2-977a-c35399f493f6 None None] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:18:31.266522 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7799a2-207d-43e2-977a-c35399f493f6 None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:18:31.266648 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7799a2-207d-43e2-977a-c35399f493f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:18:31.331116 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.auth.core [None req-ce7799a2-207d-43e2-977a-c35399f493f6 None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81449) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:18:31.332319 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-ce7799a2-207d-43e2-977a-c35399f493f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:18:31.369870 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:18:31.431255 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 735/1463] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 15 12:18:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:18:31.718126 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f3f12a-34aa-437d-bddf-6ab8fbc76642 None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:18:31.720563 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-73f3f12a-34aa-437d-bddf-6ab8fbc76642 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:18:31.828602 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f3f12a-34aa-437d-bddf-6ab8fbc76642 None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:18:31.867112 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:18:31.902158 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d956c5cd-b932-4262-8f57-69309a16e829 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfbb4d6809064f9cb9e06626ed7d6814', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '540a32ef88764e8c8d9201ae22f6ddaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:18:31.902770 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d956c5cd-b932-4262-8f57-69309a16e829 service neutron] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:18:31.903045 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d956c5cd-b932-4262-8f57-69309a16e829 service neutron] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:18:31.903311 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d956c5cd-b932-4262-8f57-69309a16e829 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:18:31.905513 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-d956c5cd-b932-4262-8f57-69309a16e829 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:18:32.013380 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d956c5cd-b932-4262-8f57-69309a16e829 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:18:32.014802 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d956c5cd-b932-4262-8f57-69309a16e829 service neutron] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:18:32.032940 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 729/1464] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 12:18:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 12:18:34.525738 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f74a22-c25a-44cd-aa0a-0b4b4872079c None None] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:18:34.526302 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f74a22-c25a-44cd-aa0a-0b4b4872079c None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:18:34.526401 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f74a22-c25a-44cd-aa0a-0b4b4872079c None None] PATH_INFO: `/` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:18:34.527940 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 736/1465] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 15 12:18:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:18:34.533292 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b913d73-f28f-429c-bfbf-e13bb4fbe6c1 None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:18:34.533735 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b913d73-f28f-429c-bfbf-e13bb4fbe6c1 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:18:34.533840 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b913d73-f28f-429c-bfbf-e13bb4fbe6c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:18:34.600714 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-1b913d73-f28f-429c-bfbf-e13bb4fbe6c1 None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:18:34.601963 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-1b913d73-f28f-429c-bfbf-e13bb4fbe6c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:18:34.620838 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:18:34.699712 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 730/1466] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 15 12:18:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:18:34.981775 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f74a22-c25a-44cd-aa0a-0b4b4872079c None None] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:18:34.983940 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-d8f74a22-c25a-44cd-aa0a-0b4b4872079c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:18:35.084877 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f74a22-c25a-44cd-aa0a-0b4b4872079c None None] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:18:35.152126 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:18:35.159803 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ecfdd84-b690-443a-ba31-8edafdfcbb2c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfbb4d6809064f9cb9e06626ed7d6814', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '540a32ef88764e8c8d9201ae22f6ddaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:18:35.160410 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecfdd84-b690-443a-ba31-8edafdfcbb2c service neutron] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:18:35.160687 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecfdd84-b690-443a-ba31-8edafdfcbb2c service neutron] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:18:35.160986 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecfdd84-b690-443a-ba31-8edafdfcbb2c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:18:35.162629 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-8ecfdd84-b690-443a-ba31-8edafdfcbb2c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:18:35.272548 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ecfdd84-b690-443a-ba31-8edafdfcbb2c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:18:35.273929 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ecfdd84-b690-443a-ba31-8edafdfcbb2c service neutron] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:18:35.292167 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 737/1467] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 12:18:34 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 12:18:37.465501 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce11c983-ec0a-4847-a0c7-2e07b084ddb0 None None] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:18:37.466088 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce11c983-ec0a-4847-a0c7-2e07b084ddb0 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:18:37.466088 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce11c983-ec0a-4847-a0c7-2e07b084ddb0 None None] PATH_INFO: `/` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:18:37.467921 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 731/1468] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 15 12:18:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:18:37.473685 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6abfcd-f9fe-49e6-89df-a69091ab7e14 None None] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:18:37.474068 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6abfcd-f9fe-49e6-89df-a69091ab7e14 None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:18:37.474234 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6abfcd-f9fe-49e6-89df-a69091ab7e14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:18:37.540304 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.auth.core [None req-8e6abfcd-f9fe-49e6-89df-a69091ab7e14 None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81449) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:18:37.541543 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-8e6abfcd-f9fe-49e6-89df-a69091ab7e14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:18:37.589169 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:18:37.638205 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 738/1469] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 15 12:18:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:18:37.966329 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce11c983-ec0a-4847-a0c7-2e07b084ddb0 None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:18:37.967884 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-ce11c983-ec0a-4847-a0c7-2e07b084ddb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:18:38.069989 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce11c983-ec0a-4847-a0c7-2e07b084ddb0 None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:18:38.119453 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:18:38.146144 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91eb9b1-b00f-43f7-81f8-496448f157ff service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfbb4d6809064f9cb9e06626ed7d6814', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '540a32ef88764e8c8d9201ae22f6ddaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:18:38.146754 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91eb9b1-b00f-43f7-81f8-496448f157ff service neutron] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:18:38.147031 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91eb9b1-b00f-43f7-81f8-496448f157ff service neutron] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:18:38.147301 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91eb9b1-b00f-43f7-81f8-496448f157ff service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:18:38.148917 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-b91eb9b1-b00f-43f7-81f8-496448f157ff service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:18:38.255947 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b91eb9b1-b00f-43f7-81f8-496448f157ff service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:18:38.257325 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b91eb9b1-b00f-43f7-81f8-496448f157ff service neutron] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:18:38.274985 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 732/1470] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 12:18:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 12:18:47.636704 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=22 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:18:48.272850 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=66 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:18:58.826478 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a7c0b8-ebc4-4463-a573-900433e45b69 None None] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:18:58.827588 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a7c0b8-ebc4-4463-a573-900433e45b69 None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:18:58.827695 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a7c0b8-ebc4-4463-a573-900433e45b69 None None] PATH_INFO: `/` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:18:58.829112 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 739/1471] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 12:18:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:18:58.834235 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689b1043-bc40-41af-a41f-e9e87e391d23 None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:18:58.834631 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689b1043-bc40-41af-a41f-e9e87e391d23 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:18:58.834785 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689b1043-bc40-41af-a41f-e9e87e391d23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:18:58.902588 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-689b1043-bc40-41af-a41f-e9e87e391d23 None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:18:58.903797 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-689b1043-bc40-41af-a41f-e9e87e391d23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:18:59.001158 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 733/1472] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 12:18:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:18:59.301031 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f97194-71e3-4bea-975c-361f9850a04e None None] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:18:59.301321 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f97194-71e3-4bea-975c-361f9850a04e None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:18:59.301599 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f97194-71e3-4bea-975c-361f9850a04e None None] PATH_INFO: `/` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:18:59.302964 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 740/1473] 10.222.0.15 () {54 vars in 962 bytes} [Thu Sep 15 12:18:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:18:59.307983 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c68d7c5-6ca0-46d1-9f9f-a83af7d70fc1 None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:18:59.308270 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c68d7c5-6ca0-46d1-9f9f-a83af7d70fc1 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:18:59.308540 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c68d7c5-6ca0-46d1-9f9f-a83af7d70fc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:18:59.383042 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-3c68d7c5-6ca0-46d1-9f9f-a83af7d70fc1 None None] MFA Rules not processed for user `3aed8f78903f44cda9e5254aa46cf2b7`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:18:59.384229 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-3c68d7c5-6ca0-46d1-9f9f-a83af7d70fc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:18:59.404104 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:18:59.459219 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 734/1474] 10.222.0.15 () {58 vars in 1059 bytes} [Thu Sep 15 12:18:59 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:18:59.466800 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f97194-71e3-4bea-975c-361f9850a04e None None] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:18:59.468303 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-b9f97194-71e3-4bea-975c-361f9850a04e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:18:59.570151 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f97194-71e3-4bea-975c-361f9850a04e None None] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:18:59.644203 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-727bb61d-bc6a-461b-8b03-1bd1c3522f79 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3aed8f78903f44cda9e5254aa46cf2b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '540a32ef88764e8c8d9201ae22f6ddaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:18:59.644833 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727bb61d-bc6a-461b-8b03-1bd1c3522f79 service glance] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:18:59.645113 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727bb61d-bc6a-461b-8b03-1bd1c3522f79 service glance] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:18:59.645382 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-727bb61d-bc6a-461b-8b03-1bd1c3522f79 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:18:59.646983 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-727bb61d-bc6a-461b-8b03-1bd1c3522f79 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:18:59.817074 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:18:59.871402 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727bb61d-bc6a-461b-8b03-1bd1c3522f79 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:18:59.872862 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-727bb61d-bc6a-461b-8b03-1bd1c3522f79 service glance] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:18:59.890829 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 741/1475] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 12:18:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 424 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 12:19:02.127108 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77db17d7-b9c0-45d2-b3db-ca85cfedea89 None None] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:19:02.127704 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77db17d7-b9c0-45d2-b3db-ca85cfedea89 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:19:02.127704 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77db17d7-b9c0-45d2-b3db-ca85cfedea89 None None] PATH_INFO: `/` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:19:02.129047 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 735/1476] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 12:19:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 15 12:19:02.134198 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39264001-713d-442c-961e-50975aed6dd3 None None] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:19:02.134633 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39264001-713d-442c-961e-50975aed6dd3 None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:19:02.134734 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39264001-713d-442c-961e-50975aed6dd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:19:02.199642 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.auth.core [None req-39264001-713d-442c-961e-50975aed6dd3 None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81449) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:19:02.200865 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-39264001-713d-442c-961e-50975aed6dd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:19:02.294306 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 742/1477] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 12:19:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:19:02.574774 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77db17d7-b9c0-45d2-b3db-ca85cfedea89 None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:19:02.576231 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-77db17d7-b9c0-45d2-b3db-ca85cfedea89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:19:02.700478 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77db17d7-b9c0-45d2-b3db-ca85cfedea89 None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:19:02.752044 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:19:02.798059 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da25588-379d-407b-bb00-c50525a354dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:19:02.798667 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da25588-379d-407b-bb00-c50525a354dd admin admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:19:02.798943 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da25588-379d-407b-bb00-c50525a354dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:19:02.799205 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da25588-379d-407b-bb00-c50525a354dd admin admin] PATH_INFO: `/v3/projects` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:19:02.799825 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da25588-379d-407b-bb00-c50525a354dd admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:19:02.801533 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: WARNING py.warnings [None req-1da25588-379d-407b-bb00-c50525a354dd admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 15 12:19:02.801533 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: warnings.warn(msg) Sep 15 12:19:02.801533 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: Sep 15 12:19:02.802658 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da25588-379d-407b-bb00-c50525a354dd admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:19:02.821396 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 736/1478] 10.222.0.15 () {56 vars in 1108 bytes} [Thu Sep 15 12:19:02 2022] GET /identity/v3/projects => generated 2553 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 12:19:04.975366 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115d8a08-8a58-4560-bf0c-4588bdb3ac01 None None] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:19:04.975915 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115d8a08-8a58-4560-bf0c-4588bdb3ac01 None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:19:04.976078 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115d8a08-8a58-4560-bf0c-4588bdb3ac01 None None] PATH_INFO: `/` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:19:04.977459 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 743/1479] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 12:19:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:19:04.982763 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048720e3-a156-48a8-821a-65176281bdbe None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:19:04.983221 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048720e3-a156-48a8-821a-65176281bdbe None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:19:04.983332 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048720e3-a156-48a8-821a-65176281bdbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:19:05.049940 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-048720e3-a156-48a8-821a-65176281bdbe None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:19:05.051125 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-048720e3-a156-48a8-821a-65176281bdbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:19:05.145679 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 737/1480] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 12:19:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 12:19:05.441598 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115d8a08-8a58-4560-bf0c-4588bdb3ac01 None None] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:19:05.443191 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-115d8a08-8a58-4560-bf0c-4588bdb3ac01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:19:05.453983 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:19:05.592105 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115d8a08-8a58-4560-bf0c-4588bdb3ac01 None None] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:19:05.672539 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:19:05.704028 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d021ac65-4ba0-4031-b1e7-3353b3db9694 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eface89e261840a2abb4206be26455c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '540a32ef88764e8c8d9201ae22f6ddaf', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:19:05.704628 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d021ac65-4ba0-4031-b1e7-3353b3db9694 service nova] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:19:05.704948 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d021ac65-4ba0-4031-b1e7-3353b3db9694 service nova] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:19:05.705214 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d021ac65-4ba0-4031-b1e7-3353b3db9694 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:19:05.706793 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-d021ac65-4ba0-4031-b1e7-3353b3db9694 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:19:05.793366 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d021ac65-4ba0-4031-b1e7-3353b3db9694 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:19:05.794953 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d021ac65-4ba0-4031-b1e7-3353b3db9694 service nova] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:19:05.813223 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 744/1481] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 12:19:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 12:19:08.106859 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a76cec-c6e4-46a6-853f-bf055ddbe126 None None] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:19:08.107455 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a76cec-c6e4-46a6-853f-bf055ddbe126 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:19:08.107562 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a76cec-c6e4-46a6-853f-bf055ddbe126 None None] PATH_INFO: `/` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:19:08.109045 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 738/1482] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 12:19:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:19:08.114270 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1924459-7707-4ea1-86bc-e12182714d7f None None] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:19:08.114689 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1924459-7707-4ea1-86bc-e12182714d7f None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:19:08.114882 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1924459-7707-4ea1-86bc-e12182714d7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:19:08.180137 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.auth.core [None req-b1924459-7707-4ea1-86bc-e12182714d7f None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81449) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:19:08.181361 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-b1924459-7707-4ea1-86bc-e12182714d7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:19:08.226838 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:19:08.275911 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 745/1483] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 12:19:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:19:08.560711 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a76cec-c6e4-46a6-853f-bf055ddbe126 None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:19:08.562906 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-c0a76cec-c6e4-46a6-853f-bf055ddbe126 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:19:08.586094 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:19:08.667691 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a76cec-c6e4-46a6-853f-bf055ddbe126 None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:19:08.742597 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9761e8-85f4-4edc-ab04-1f0103061b44 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfbb4d6809064f9cb9e06626ed7d6814', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '540a32ef88764e8c8d9201ae22f6ddaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:19:08.743196 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9761e8-85f4-4edc-ab04-1f0103061b44 service neutron] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:19:08.743470 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9761e8-85f4-4edc-ab04-1f0103061b44 service neutron] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:19:08.743735 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9761e8-85f4-4edc-ab04-1f0103061b44 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:19:08.745337 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-8a9761e8-85f4-4edc-ab04-1f0103061b44 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:19:08.818239 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:19:08.855951 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9761e8-85f4-4edc-ab04-1f0103061b44 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:19:08.857448 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9761e8-85f4-4edc-ab04-1f0103061b44 service neutron] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:19:08.875935 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 739/1484] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 12:19:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 12:19:11.087662 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1de929d-d5ec-42a5-ba98-9a1ade90cb95 None None] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:19:11.088238 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1de929d-d5ec-42a5-ba98-9a1ade90cb95 None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:19:11.088238 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1de929d-d5ec-42a5-ba98-9a1ade90cb95 None None] PATH_INFO: `/` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:19:11.089600 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 746/1485] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 12:19:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:19:11.094889 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a743b8-bfa4-4b68-b5b1-4f925f5a3daa None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:19:11.095353 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a743b8-bfa4-4b68-b5b1-4f925f5a3daa None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:19:11.095467 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a743b8-bfa4-4b68-b5b1-4f925f5a3daa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:19:11.166440 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-87a743b8-bfa4-4b68-b5b1-4f925f5a3daa None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:19:11.167669 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-87a743b8-bfa4-4b68-b5b1-4f925f5a3daa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:19:11.264600 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 740/1486] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 12:19:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:19:11.557087 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1de929d-d5ec-42a5-ba98-9a1ade90cb95 None None] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:19:11.558648 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-b1de929d-d5ec-42a5-ba98-9a1ade90cb95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:19:11.660632 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1de929d-d5ec-42a5-ba98-9a1ade90cb95 None None] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:19:11.730961 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:19:11.732746 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7505c7-fe44-451b-b362-cfa335630747 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfbb4d6809064f9cb9e06626ed7d6814', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '540a32ef88764e8c8d9201ae22f6ddaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:19:11.733436 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7505c7-fe44-451b-b362-cfa335630747 service neutron] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:19:11.733723 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7505c7-fe44-451b-b362-cfa335630747 service neutron] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:19:11.734004 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7505c7-fe44-451b-b362-cfa335630747 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:19:11.736794 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-8d7505c7-fe44-451b-b362-cfa335630747 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:19:11.845363 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d7505c7-fe44-451b-b362-cfa335630747 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:19:11.846964 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d7505c7-fe44-451b-b362-cfa335630747 service neutron] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:19:11.864894 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 747/1487] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 12:19:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 12:19:14.197244 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644e9d33-f344-4711-99ac-9c850dcd61ca None None] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:19:14.197815 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644e9d33-f344-4711-99ac-9c850dcd61ca None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:19:14.197815 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644e9d33-f344-4711-99ac-9c850dcd61ca None None] PATH_INFO: `/` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:19:14.199128 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 741/1488] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 12:19:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:19:14.205090 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3f6c99-f48c-4d11-8f92-396b0230b447 None None] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:19:14.205522 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3f6c99-f48c-4d11-8f92-396b0230b447 None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:19:14.205713 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3f6c99-f48c-4d11-8f92-396b0230b447 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:19:14.270265 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.auth.core [None req-9b3f6c99-f48c-4d11-8f92-396b0230b447 None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81449) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:19:14.271416 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-9b3f6c99-f48c-4d11-8f92-396b0230b447 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:19:14.322341 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:19:14.366614 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 748/1489] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 12:19:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:19:14.665225 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644e9d33-f344-4711-99ac-9c850dcd61ca None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:19:14.666757 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-644e9d33-f344-4711-99ac-9c850dcd61ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:19:14.703735 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:19:14.772934 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644e9d33-f344-4711-99ac-9c850dcd61ca None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:19:14.844698 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e785cfe-ad67-407d-87d7-2f7886fdc003 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfbb4d6809064f9cb9e06626ed7d6814', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '540a32ef88764e8c8d9201ae22f6ddaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:19:14.845393 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e785cfe-ad67-407d-87d7-2f7886fdc003 service neutron] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:19:14.845627 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e785cfe-ad67-407d-87d7-2f7886fdc003 service neutron] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:19:14.845894 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e785cfe-ad67-407d-87d7-2f7886fdc003 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:19:14.847719 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-2e785cfe-ad67-407d-87d7-2f7886fdc003 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:19:14.926355 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:19:14.956740 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e785cfe-ad67-407d-87d7-2f7886fdc003 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:19:14.956740 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e785cfe-ad67-407d-87d7-2f7886fdc003 service neutron] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:19:14.974778 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 742/1490] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 12:19:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 12:19:17.345480 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9618d7-2041-4ee2-8ec1-469bca5f7b80 None None] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:19:17.346014 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9618d7-2041-4ee2-8ec1-469bca5f7b80 None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:19:17.346014 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9618d7-2041-4ee2-8ec1-469bca5f7b80 None None] PATH_INFO: `/` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:19:17.347538 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 749/1491] 10.222.0.15 () {54 vars in 927 bytes} [Thu Sep 15 12:19:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:19:17.352861 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faee8eaf-6bc0-416e-8813-46990596a5d6 None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:19:17.353259 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faee8eaf-6bc0-416e-8813-46990596a5d6 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:19:17.353414 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faee8eaf-6bc0-416e-8813-46990596a5d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:19:17.419106 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-faee8eaf-6bc0-416e-8813-46990596a5d6 None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:19:17.420317 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-faee8eaf-6bc0-416e-8813-46990596a5d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:19:17.514763 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 743/1492] 10.222.0.15 () {58 vars in 1024 bytes} [Thu Sep 15 12:19:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:19:17.805132 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9618d7-2041-4ee2-8ec1-469bca5f7b80 None None] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:19:17.806647 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-ce9618d7-2041-4ee2-8ec1-469bca5f7b80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:19:17.908360 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9618d7-2041-4ee2-8ec1-469bca5f7b80 None None] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:19:17.986182 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223a9cf3-17ea-4371-85aa-1033678ba139 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfbb4d6809064f9cb9e06626ed7d6814', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '540a32ef88764e8c8d9201ae22f6ddaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:19:17.986786 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223a9cf3-17ea-4371-85aa-1033678ba139 service neutron] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:19:17.987064 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223a9cf3-17ea-4371-85aa-1033678ba139 service neutron] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:19:17.987332 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223a9cf3-17ea-4371-85aa-1033678ba139 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:19:17.988942 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-223a9cf3-17ea-4371-85aa-1033678ba139 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:19:17.997368 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:19:18.099000 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-223a9cf3-17ea-4371-85aa-1033678ba139 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:19:18.100361 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-223a9cf3-17ea-4371-85aa-1033678ba139 service neutron] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:19:18.119082 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 750/1493] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 12:19:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 12:19:27.512727 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=82 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:19:28.116837 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=50 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:20:36.526155 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec326bf6-2204-412a-8fb8-030746027d63 None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:20:36.526755 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec326bf6-2204-412a-8fb8-030746027d63 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:20:36.526755 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec326bf6-2204-412a-8fb8-030746027d63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:20:36.602736 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-ec326bf6-2204-412a-8fb8-030746027d63 None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:20:36.603944 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-ec326bf6-2204-412a-8fb8-030746027d63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:20:36.698076 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 744/1494] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 12:20:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:20:36.706572 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ddba68-efe0-4a24-b138-a886afc61f45 None None] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:20:36.707099 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ddba68-efe0-4a24-b138-a886afc61f45 None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:20:36.707099 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ddba68-efe0-4a24-b138-a886afc61f45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:20:36.781889 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.auth.core [None req-f7ddba68-efe0-4a24-b138-a886afc61f45 None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81449) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:20:36.783013 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-f7ddba68-efe0-4a24-b138-a886afc61f45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:20:36.876302 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 751/1495] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 12:20:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:20:36.881013 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec326bf6-2204-412a-8fb8-030746027d63 None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:20:36.882191 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-ec326bf6-2204-412a-8fb8-030746027d63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:20:36.893424 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec326bf6-2204-412a-8fb8-030746027d63 None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:20:36.952890 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:20:36.987405 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38d3518-0947-40c5-9038-421dcf64fdbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:20:36.988010 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38d3518-0947-40c5-9038-421dcf64fdbf admin admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:20:36.988286 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38d3518-0947-40c5-9038-421dcf64fdbf admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:20:36.988548 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38d3518-0947-40c5-9038-421dcf64fdbf admin admin] PATH_INFO: `/v3/domains` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:20:36.989211 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e38d3518-0947-40c5-9038-421dcf64fdbf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:20:36.989508 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e38d3518-0947-40c5-9038-421dcf64fdbf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:20:36.990531 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: WARNING py.warnings [None req-e38d3518-0947-40c5-9038-421dcf64fdbf admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 15 12:20:36.990531 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: warnings.warn(msg) Sep 15 12:20:36.990531 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: Sep 15 12:20:36.991398 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e38d3518-0947-40c5-9038-421dcf64fdbf admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:20:37.002995 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 745/1496] 10.222.0.15 () {58 vars in 1153 bytes} [Thu Sep 15 12:20:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 122 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:20:37.011238 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ddba68-efe0-4a24-b138-a886afc61f45 None None] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:20:37.020172 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ddba68-efe0-4a24-b138-a886afc61f45 None None] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:20:37.081028 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:20:37.113535 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18362e9f-4570-47a8-aad7-1397fa3187fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:20:37.114139 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18362e9f-4570-47a8-aad7-1397fa3187fb admin admin] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:20:37.114412 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18362e9f-4570-47a8-aad7-1397fa3187fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:20:37.114674 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18362e9f-4570-47a8-aad7-1397fa3187fb admin admin] PATH_INFO: `/v3/projects` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:20:37.115287 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18362e9f-4570-47a8-aad7-1397fa3187fb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:20:37.117028 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: WARNING py.warnings [None req-18362e9f-4570-47a8-aad7-1397fa3187fb admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 15 12:20:37.117028 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: warnings.warn(msg) Sep 15 12:20:37.117028 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: Sep 15 12:20:37.118205 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18362e9f-4570-47a8-aad7-1397fa3187fb admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:20:37.174246 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 752/1497] 10.222.0.15 () {60 vars in 1153 bytes} [Thu Sep 15 12:20:37 2022] POST /identity/v3/projects => generated 377 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 12:20:37.178383 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38d3518-0947-40c5-9038-421dcf64fdbf admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:20:37.179881 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-e38d3518-0947-40c5-9038-421dcf64fdbf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:20:37.213330 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38d3518-0947-40c5-9038-421dcf64fdbf admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:20:37.308207 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e60a93b-90f6-4173-bf00-f81b0328e480 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:20:37.308838 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e60a93b-90f6-4173-bf00-f81b0328e480 admin admin] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:20:37.309116 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e60a93b-90f6-4173-bf00-f81b0328e480 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:20:37.309380 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e60a93b-90f6-4173-bf00-f81b0328e480 admin admin] PATH_INFO: `/v3/users` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:20:37.310010 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e60a93b-90f6-4173-bf00-f81b0328e480 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:20:37.311039 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: WARNING py.warnings [None req-0e60a93b-90f6-4173-bf00-f81b0328e480 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 15 12:20:37.311039 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: warnings.warn(msg) Sep 15 12:20:37.311039 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: Sep 15 12:20:37.312224 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e60a93b-90f6-4173-bf00-f81b0328e480 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:20:37.352975 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 746/1498] 10.222.0.15 () {60 vars in 1144 bytes} [Thu Sep 15 12:20:37 2022] POST /identity/v3/users => generated 289 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 15 12:20:37.357945 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18362e9f-4570-47a8-aad7-1397fa3187fb admin admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:20:37.359447 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-18362e9f-4570-47a8-aad7-1397fa3187fb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:20:37.419276 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18362e9f-4570-47a8-aad7-1397fa3187fb admin admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:20:37.501143 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:20:37.512555 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a94978-0cfd-47aa-827d-658e71cde784 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:20:37.513196 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a94978-0cfd-47aa-827d-658e71cde784 admin admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:20:37.513988 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a94978-0cfd-47aa-827d-658e71cde784 admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:20:37.514255 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a94978-0cfd-47aa-827d-658e71cde784 admin admin] PATH_INFO: `/v3/roles` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:20:37.514884 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a94978-0cfd-47aa-827d-658e71cde784 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:20:37.515887 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: WARNING py.warnings [None req-e7a94978-0cfd-47aa-827d-658e71cde784 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 15 12:20:37.515887 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: warnings.warn(msg) Sep 15 12:20:37.515887 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: Sep 15 12:20:37.516784 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a94978-0cfd-47aa-827d-658e71cde784 admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:20:37.528911 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 753/1499] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 12:20:37 2022] GET /identity/v3/roles => generated 1420 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 12:20:37.533425 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e60a93b-90f6-4173-bf00-f81b0328e480 admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:20:37.534954 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-0e60a93b-90f6-4173-bf00-f81b0328e480 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:20:37.546338 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e60a93b-90f6-4173-bf00-f81b0328e480 admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:20:37.570905 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:20:37.645872 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d19eff2-7ae4-437d-8e9b-9c2178ef33b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:20:37.646403 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d19eff2-7ae4-437d-8e9b-9c2178ef33b0 admin admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:20:37.646679 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d19eff2-7ae4-437d-8e9b-9c2178ef33b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:20:37.646946 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d19eff2-7ae4-437d-8e9b-9c2178ef33b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:20:37.647577 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d19eff2-7ae4-437d-8e9b-9c2178ef33b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:20:37.648578 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: WARNING py.warnings [None req-7d19eff2-7ae4-437d-8e9b-9c2178ef33b0 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 15 12:20:37.648578 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: warnings.warn(msg) Sep 15 12:20:37.648578 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: Sep 15 12:20:37.649482 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d19eff2-7ae4-437d-8e9b-9c2178ef33b0 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:20:37.662004 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 747/1500] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 12:20:37 2022] GET /identity/v3/roles => generated 1420 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 12:20:37.666140 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a94978-0cfd-47aa-827d-658e71cde784 admin admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:20:37.667639 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-e7a94978-0cfd-47aa-827d-658e71cde784 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:20:37.699853 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a94978-0cfd-47aa-827d-658e71cde784 admin admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:20:37.792121 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f224a709-d5ec-4beb-b3d4-9c710cfbd701 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:20:37.792820 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f224a709-d5ec-4beb-b3d4-9c710cfbd701 admin admin] REQUEST_METHOD: `PUT` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:20:37.793063 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f224a709-d5ec-4beb-b3d4-9c710cfbd701 admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:20:37.793330 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f224a709-d5ec-4beb-b3d4-9c710cfbd701 admin admin] PATH_INFO: `/v3/projects/d2016dd8835c46c29a2b2cb9aec7f9a0/users/ddfde7e89cd94f178669ba3b38ae94fb/roles/0679d6628a25438d90dc4bdc0401eda2` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:20:37.815740 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f224a709-d5ec-4beb-b3d4-9c710cfbd701 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d2016dd8835c46c29a2b2cb9aec7f9a0, user_id=ddfde7e89cd94f178669ba3b38ae94fb, role_id=0679d6628a25438d90dc4bdc0401eda2)` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:20:37.816773 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: WARNING py.warnings [None req-f224a709-d5ec-4beb-b3d4-9c710cfbd701 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 15 12:20:37.816773 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: warnings.warn(msg) Sep 15 12:20:37.816773 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: Sep 15 12:20:37.818712 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f224a709-d5ec-4beb-b3d4-9c710cfbd701 admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:20:37.842607 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 754/1501] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 15 12:20:37 2022] PUT /identity/v3/projects/d2016dd8835c46c29a2b2cb9aec7f9a0/users/ddfde7e89cd94f178669ba3b38ae94fb/roles/0679d6628a25438d90dc4bdc0401eda2 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:20:37.855211 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ca0dbd-ed9d-48d2-b643-6dd462994c8a None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:20:37.855633 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ca0dbd-ed9d-48d2-b643-6dd462994c8a None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:20:37.855912 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ca0dbd-ed9d-48d2-b643-6dd462994c8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:20:37.912147 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-72ca0dbd-ed9d-48d2-b643-6dd462994c8a None None] MFA Rules not processed for user `ddfde7e89cd94f178669ba3b38ae94fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:20:37.913301 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-72ca0dbd-ed9d-48d2-b643-6dd462994c8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:20:37.989213 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 748/1502] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 12:20:37 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:20:38.002691 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c09a026-0c15-42b7-a738-9a2f45946bd0 None None] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:20:38.002975 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c09a026-0c15-42b7-a738-9a2f45946bd0 None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:20:38.003265 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c09a026-0c15-42b7-a738-9a2f45946bd0 None None] PATH_INFO: `/` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:20:38.004641 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 755/1503] 10.222.0.15 () {54 vars in 964 bytes} [Thu Sep 15 12:20:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:20:38.009582 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4bd10f-6787-4462-8f2d-70b9495b493e None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:20:38.009861 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4bd10f-6787-4462-8f2d-70b9495b493e None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:20:38.010127 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4bd10f-6787-4462-8f2d-70b9495b493e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:20:38.041543 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:20:38.086025 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-fd4bd10f-6787-4462-8f2d-70b9495b493e None None] MFA Rules not processed for user `c8191d3ccb874529b32f8d0026db629d`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:20:38.087233 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-fd4bd10f-6787-4462-8f2d-70b9495b493e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:20:38.162832 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 749/1504] 10.222.0.15 () {58 vars in 1061 bytes} [Thu Sep 15 12:20:38 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:20:38.171256 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c09a026-0c15-42b7-a738-9a2f45946bd0 None None] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:20:38.172804 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-3c09a026-0c15-42b7-a738-9a2f45946bd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:20:38.222762 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:20:38.276140 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c09a026-0c15-42b7-a738-9a2f45946bd0 None None] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:20:38.348428 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6bc0c4-9c0f-40c5-b08c-94f5dd451b06 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8191d3ccb874529b32f8d0026db629d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '540a32ef88764e8c8d9201ae22f6ddaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:20:38.349059 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6bc0c4-9c0f-40c5-b08c-94f5dd451b06 service cinder] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:20:38.349336 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6bc0c4-9c0f-40c5-b08c-94f5dd451b06 service cinder] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:20:38.349602 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6bc0c4-9c0f-40c5-b08c-94f5dd451b06 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:20:38.351177 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-1b6bc0c4-9c0f-40c5-b08c-94f5dd451b06 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:20:38.446762 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6bc0c4-9c0f-40c5-b08c-94f5dd451b06 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:20:38.448182 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6bc0c4-9c0f-40c5-b08c-94f5dd451b06 service cinder] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:20:38.452144 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:20:38.466996 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 756/1505] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 12:20:38 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 12:20:38.511254 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8eeb83-a420-44e7-b915-db78ca47a285 None None] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:20:38.511540 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8eeb83-a420-44e7-b915-db78ca47a285 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:20:38.511814 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8eeb83-a420-44e7-b915-db78ca47a285 None None] PATH_INFO: `/` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:20:38.513121 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 750/1506] 10.222.0.15 () {52 vars in 830 bytes} [Thu Sep 15 12:20:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:20:38.517191 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6bc0c4-9c0f-40c5-b08c-94f5dd451b06 service cinder] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:20:38.518422 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-1b6bc0c4-9c0f-40c5-b08c-94f5dd451b06 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:20:38.615393 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6bc0c4-9c0f-40c5-b08c-94f5dd451b06 service cinder] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:20:38.706055 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a200a680-0e28-471a-bf4b-4b9e862981b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:20:38.706601 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a200a680-0e28-471a-bf4b-4b9e862981b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:20:38.706875 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a200a680-0e28-471a-bf4b-4b9e862981b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:20:38.707146 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a200a680-0e28-471a-bf4b-4b9e862981b0 admin admin] PATH_INFO: `/v3/users/ddfde7e89cd94f178669ba3b38ae94fb` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:20:38.720050 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a200a680-0e28-471a-bf4b-4b9e862981b0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ddfde7e89cd94f178669ba3b38ae94fb)` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:20:38.721092 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: WARNING py.warnings [None req-a200a680-0e28-471a-bf4b-4b9e862981b0 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 15 12:20:38.721092 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: warnings.warn(msg) Sep 15 12:20:38.721092 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: Sep 15 12:20:38.722277 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a200a680-0e28-471a-bf4b-4b9e862981b0 admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:20:38.731932 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:20:38.787522 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-a200a680-0e28-471a-bf4b-4b9e862981b0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ddfde7e89cd94f178669ba3b38ae94fb'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:20:38.797279 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-a200a680-0e28-471a-bf4b-4b9e862981b0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ddfde7e89cd94f178669ba3b38ae94fb'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:20:38.820171 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-a200a680-0e28-471a-bf4b-4b9e862981b0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ddfde7e89cd94f178669ba3b38ae94fb'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:20:38.820611 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-a200a680-0e28-471a-bf4b-4b9e862981b0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ddfde7e89cd94f178669ba3b38ae94fb'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:20:38.833973 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-a200a680-0e28-471a-bf4b-4b9e862981b0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ddfde7e89cd94f178669ba3b38ae94fb'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:20:38.839421 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 757/1507] 10.222.0.15 () {58 vars in 1224 bytes} [Thu Sep 15 12:20:38 2022] DELETE /identity/v3/users/ddfde7e89cd94f178669ba3b38ae94fb => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:20:38.844262 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428be095-5d6d-4e05-93ef-8976b9454fc7 None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:20:38.844725 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428be095-5d6d-4e05-93ef-8976b9454fc7 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:20:38.844857 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428be095-5d6d-4e05-93ef-8976b9454fc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:20:38.900864 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-428be095-5d6d-4e05-93ef-8976b9454fc7 None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:20:38.902085 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-428be095-5d6d-4e05-93ef-8976b9454fc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:20:38.978492 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:20:38.991724 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 751/1508] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 12:20:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:20:39.001253 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a200a680-0e28-471a-bf4b-4b9e862981b0 admin admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:20:39.002481 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-a200a680-0e28-471a-bf4b-4b9e862981b0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:20:39.104607 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a200a680-0e28-471a-bf4b-4b9e862981b0 admin admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:20:39.158207 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:20:39.178813 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58068c96-184f-4fac-b459-84ceaeaf6332 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfbb4d6809064f9cb9e06626ed7d6814', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '540a32ef88764e8c8d9201ae22f6ddaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:20:39.179429 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58068c96-184f-4fac-b459-84ceaeaf6332 service neutron] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:20:39.179709 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58068c96-184f-4fac-b459-84ceaeaf6332 service neutron] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:20:39.179981 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58068c96-184f-4fac-b459-84ceaeaf6332 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:20:39.181613 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-58068c96-184f-4fac-b459-84ceaeaf6332 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:20:39.289719 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58068c96-184f-4fac-b459-84ceaeaf6332 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:20:39.291066 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58068c96-184f-4fac-b459-84ceaeaf6332 service neutron] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:20:39.308433 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 758/1509] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 12:20:39 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 12:20:39.630420 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428be095-5d6d-4e05-93ef-8976b9454fc7 None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:20:39.631673 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-428be095-5d6d-4e05-93ef-8976b9454fc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:20:39.643389 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428be095-5d6d-4e05-93ef-8976b9454fc7 None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:20:39.748520 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da57dddd-3562-4333-913b-ffd949b22002 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:20:39.749191 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da57dddd-3562-4333-913b-ffd949b22002 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:20:39.749464 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da57dddd-3562-4333-913b-ffd949b22002 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:20:39.749730 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da57dddd-3562-4333-913b-ffd949b22002 admin admin] PATH_INFO: `/v3/projects/d2016dd8835c46c29a2b2cb9aec7f9a0` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:20:39.759329 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da57dddd-3562-4333-913b-ffd949b22002 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d2016dd8835c46c29a2b2cb9aec7f9a0)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:20:39.760447 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: WARNING py.warnings [None req-da57dddd-3562-4333-913b-ffd949b22002 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 15 12:20:39.760447 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: warnings.warn(msg) Sep 15 12:20:39.760447 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: Sep 15 12:20:39.761665 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da57dddd-3562-4333-913b-ffd949b22002 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:20:39.818241 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-da57dddd-3562-4333-913b-ffd949b22002 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd2016dd8835c46c29a2b2cb9aec7f9a0', 'request_id': 'req-da57dddd-3562-4333-913b-ffd949b22002'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:20:39.827573 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-da57dddd-3562-4333-913b-ffd949b22002 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd2016dd8835c46c29a2b2cb9aec7f9a0', 'request_id': 'req-da57dddd-3562-4333-913b-ffd949b22002'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:20:39.833642 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-da57dddd-3562-4333-913b-ffd949b22002 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd2016dd8835c46c29a2b2cb9aec7f9a0', 'request_id': 'req-da57dddd-3562-4333-913b-ffd949b22002'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:20:39.862804 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-da57dddd-3562-4333-913b-ffd949b22002 admin admin] The token cache is being invalidate because project d2016dd8835c46c29a2b2cb9aec7f9a0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81448) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 12:20:39.863390 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-da57dddd-3562-4333-913b-ffd949b22002 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:20:39.863589 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-da57dddd-3562-4333-913b-ffd949b22002 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:20:39.865008 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 752/1510] 10.222.0.15 () {58 vars in 1233 bytes} [Thu Sep 15 12:20:39 2022] DELETE /identity/v3/projects/d2016dd8835c46c29a2b2cb9aec7f9a0 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:20:49.306308 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=62 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:20:49.860745 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:22:09.795378 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdd1945-a249-4cee-a07c-8df79b17fc7b None None] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:22:09.795946 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdd1945-a249-4cee-a07c-8df79b17fc7b None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:22:09.795946 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fdd1945-a249-4cee-a07c-8df79b17fc7b None None] PATH_INFO: `/` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:22:09.797770 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 759/1511] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 15 12:22:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:22:09.802507 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954fde07-92d8-402c-b6b9-d32004f24aea None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:22:09.802947 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954fde07-92d8-402c-b6b9-d32004f24aea None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:22:09.803059 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954fde07-92d8-402c-b6b9-d32004f24aea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:22:09.883349 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-954fde07-92d8-402c-b6b9-d32004f24aea None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:22:09.884614 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-954fde07-92d8-402c-b6b9-d32004f24aea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:22:09.985989 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 753/1512] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 15 12:22:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:22:10.145497 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fdd1945-a249-4cee-a07c-8df79b17fc7b None None] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:22:10.147621 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-9fdd1945-a249-4cee-a07c-8df79b17fc7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:22:10.271114 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fdd1945-a249-4cee-a07c-8df79b17fc7b None None] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:22:10.276553 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24c2908-87b3-4cd6-aba7-efead48efcc9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eface89e261840a2abb4206be26455c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '540a32ef88764e8c8d9201ae22f6ddaf', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:22:10.277183 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24c2908-87b3-4cd6-aba7-efead48efcc9 service nova] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:22:10.277461 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24c2908-87b3-4cd6-aba7-efead48efcc9 service nova] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:22:10.277728 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24c2908-87b3-4cd6-aba7-efead48efcc9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:22:10.286194 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f24c2908-87b3-4cd6-aba7-efead48efcc9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:22:10.287780 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f24c2908-87b3-4cd6-aba7-efead48efcc9 service nova] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:22:10.296909 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 760/1513] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 12:22:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 12:22:11.373609 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f991f8ba-e871-44aa-8faa-8225d720dfe0 None None] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:22:11.374161 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f991f8ba-e871-44aa-8faa-8225d720dfe0 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:22:11.374161 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f991f8ba-e871-44aa-8faa-8225d720dfe0 None None] PATH_INFO: `/` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:22:11.375976 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 754/1514] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 15 12:22:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:22:11.381962 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af589f37-0f5a-41ff-b096-ae402d52b064 None None] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:22:11.382405 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af589f37-0f5a-41ff-b096-ae402d52b064 None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:22:11.382508 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af589f37-0f5a-41ff-b096-ae402d52b064 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:22:11.401985 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.auth.core [None req-af589f37-0f5a-41ff-b096-ae402d52b064 None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81449) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:22:11.403210 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-af589f37-0f5a-41ff-b096-ae402d52b064 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:22:11.411595 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 761/1515] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 15 12:22:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:22:11.565707 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f991f8ba-e871-44aa-8faa-8225d720dfe0 None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:22:11.580140 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f991f8ba-e871-44aa-8faa-8225d720dfe0 None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:22:11.585693 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32e6320-9b60-4c06-9cea-2dcbc37f777e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eface89e261840a2abb4206be26455c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '540a32ef88764e8c8d9201ae22f6ddaf', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:22:11.586317 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32e6320-9b60-4c06-9cea-2dcbc37f777e service nova] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:22:11.586595 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32e6320-9b60-4c06-9cea-2dcbc37f777e service nova] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:22:11.586861 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32e6320-9b60-4c06-9cea-2dcbc37f777e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:22:11.595968 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32e6320-9b60-4c06-9cea-2dcbc37f777e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:22:11.597832 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32e6320-9b60-4c06-9cea-2dcbc37f777e service nova] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:22:11.607053 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 755/1516] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 12:22:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 12:22:11.617198 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63b88e2-a324-4f6b-968d-26bc5dc7ffcd None None] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:22:11.617411 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63b88e2-a324-4f6b-968d-26bc5dc7ffcd None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:22:11.617688 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63b88e2-a324-4f6b-968d-26bc5dc7ffcd None None] PATH_INFO: `/v3` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:22:11.619016 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 762/1517] 10.222.0.26 () {54 vars in 925 bytes} [Thu Sep 15 12:22:11 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:22:11.625407 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fba3b7d-b267-41fe-98ef-d5fdfcdec69c None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:22:11.625741 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fba3b7d-b267-41fe-98ef-d5fdfcdec69c None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:22:11.626021 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fba3b7d-b267-41fe-98ef-d5fdfcdec69c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:22:11.690971 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-5fba3b7d-b267-41fe-98ef-d5fdfcdec69c None None] MFA Rules not processed for user `51f64a058bd5440cb5268f228aa6e392`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:22:11.692199 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-5fba3b7d-b267-41fe-98ef-d5fdfcdec69c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:22:11.710137 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:22:11.768479 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 756/1518] 10.222.0.26 () {58 vars in 1015 bytes} [Thu Sep 15 12:22:11 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 12:22:15.340673 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d572e0-c483-49ea-b985-f46bfa34d2ff None None] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:22:15.341161 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d572e0-c483-49ea-b985-f46bfa34d2ff None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:22:15.341262 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d572e0-c483-49ea-b985-f46bfa34d2ff None None] PATH_INFO: `/` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:22:15.342564 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 763/1519] 10.222.0.15 () {54 vars in 961 bytes} [Thu Sep 15 12:22:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:22:15.347944 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d694f2-e292-4742-a16b-626b718def4c None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:22:15.348380 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d694f2-e292-4742-a16b-626b718def4c None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:22:15.348485 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d694f2-e292-4742-a16b-626b718def4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:22:15.367720 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-33d694f2-e292-4742-a16b-626b718def4c None None] MFA Rules not processed for user `51f64a058bd5440cb5268f228aa6e392`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:22:15.368936 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-33d694f2-e292-4742-a16b-626b718def4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:22:15.377786 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 757/1520] 10.222.0.15 () {58 vars in 1058 bytes} [Thu Sep 15 12:22:15 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:22:15.385184 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d572e0-c483-49ea-b985-f46bfa34d2ff None None] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:22:15.398646 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d572e0-c483-49ea-b985-f46bfa34d2ff None None] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:22:15.403462 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978bf9fd-60b1-4577-af33-23d1b92e728b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f64a058bd5440cb5268f228aa6e392', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '540a32ef88764e8c8d9201ae22f6ddaf', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:22:15.404044 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978bf9fd-60b1-4577-af33-23d1b92e728b service placement] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:22:15.404321 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978bf9fd-60b1-4577-af33-23d1b92e728b service placement] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:22:15.404610 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978bf9fd-60b1-4577-af33-23d1b92e728b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:22:15.412655 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-978bf9fd-60b1-4577-af33-23d1b92e728b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:22:15.414252 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-978bf9fd-60b1-4577-af33-23d1b92e728b service placement] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:22:15.423531 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 764/1521] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 12:22:15 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 12:22:23.091888 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f009594-cc08-4a0f-a286-278551c3b0e3 None None] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:22:23.092453 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f009594-cc08-4a0f-a286-278551c3b0e3 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:22:23.092453 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f009594-cc08-4a0f-a286-278551c3b0e3 None None] PATH_INFO: `/` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:22:23.094430 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 758/1522] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 15 12:22:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:22:23.100701 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb3085b-681d-405a-8093-32351bcac85b None None] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:22:23.101174 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb3085b-681d-405a-8093-32351bcac85b None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:22:23.101280 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb3085b-681d-405a-8093-32351bcac85b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:22:23.120242 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.auth.core [None req-6cb3085b-681d-405a-8093-32351bcac85b None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81449) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:22:23.121450 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-6cb3085b-681d-405a-8093-32351bcac85b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:22:23.129092 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 765/1523] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 15 12:22:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:22:23.280721 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f009594-cc08-4a0f-a286-278551c3b0e3 None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:22:23.295222 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f009594-cc08-4a0f-a286-278551c3b0e3 None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:22:23.300527 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded3aaa2-ee8b-4937-80c2-01512d1f672f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eface89e261840a2abb4206be26455c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '540a32ef88764e8c8d9201ae22f6ddaf', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:22:23.301153 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded3aaa2-ee8b-4937-80c2-01512d1f672f service nova] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:22:23.301431 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded3aaa2-ee8b-4937-80c2-01512d1f672f service nova] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:22:23.301699 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded3aaa2-ee8b-4937-80c2-01512d1f672f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:22:23.309487 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded3aaa2-ee8b-4937-80c2-01512d1f672f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:22:23.311037 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded3aaa2-ee8b-4937-80c2-01512d1f672f service nova] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:22:23.320079 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 759/1524] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 12:22:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 12:22:24.391268 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952deeae-a14f-42c7-accc-8ada96a6cec9 None None] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:22:24.391804 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952deeae-a14f-42c7-accc-8ada96a6cec9 None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:22:24.391804 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952deeae-a14f-42c7-accc-8ada96a6cec9 None None] PATH_INFO: `/` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:22:24.393090 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 766/1525] 127.0.0.1 () {54 vars in 901 bytes} [Thu Sep 15 12:22:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:22:24.399121 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe22cd3-f2c0-44f0-a2e4-2c8633bf32f2 None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:22:24.399552 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe22cd3-f2c0-44f0-a2e4-2c8633bf32f2 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:22:24.399685 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe22cd3-f2c0-44f0-a2e4-2c8633bf32f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:22:24.418755 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-8fe22cd3-f2c0-44f0-a2e4-2c8633bf32f2 None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:22:24.419927 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-8fe22cd3-f2c0-44f0-a2e4-2c8633bf32f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:22:24.427604 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 760/1526] 127.0.0.1 () {58 vars in 998 bytes} [Thu Sep 15 12:22:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 12:22:24.579065 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952deeae-a14f-42c7-accc-8ada96a6cec9 None None] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:22:24.592841 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952deeae-a14f-42c7-accc-8ada96a6cec9 None None] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:22:24.598358 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e35893b-8272-406d-9992-893545a967ed service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eface89e261840a2abb4206be26455c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '540a32ef88764e8c8d9201ae22f6ddaf', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:22:24.598944 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e35893b-8272-406d-9992-893545a967ed service nova] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:22:24.599221 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e35893b-8272-406d-9992-893545a967ed service nova] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:22:24.599491 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e35893b-8272-406d-9992-893545a967ed service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:22:24.607461 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e35893b-8272-406d-9992-893545a967ed service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:22:24.609591 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e35893b-8272-406d-9992-893545a967ed service nova] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:22:24.619574 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 767/1527] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 12:22:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 12:22:28.656982 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a9c874-e71f-43e9-9b61-15b3a635c82f None None] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:22:28.657590 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a9c874-e71f-43e9-9b61-15b3a635c82f None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:22:28.657590 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a9c874-e71f-43e9-9b61-15b3a635c82f None None] PATH_INFO: `/` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:22:28.658861 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 761/1528] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 15 12:22:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:22:28.664109 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5c0596-dd4f-414f-b6e5-a2acaa91a0d1 None None] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:22:28.664559 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5c0596-dd4f-414f-b6e5-a2acaa91a0d1 None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:22:28.664668 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d5c0596-dd4f-414f-b6e5-a2acaa91a0d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:22:28.684032 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.auth.core [None req-3d5c0596-dd4f-414f-b6e5-a2acaa91a0d1 None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81449) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:22:28.685242 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-3d5c0596-dd4f-414f-b6e5-a2acaa91a0d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:22:28.692846 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 768/1529] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 15 12:22:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:22:28.992286 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a9c874-e71f-43e9-9b61-15b3a635c82f None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:22:28.994021 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-89a9c874-e71f-43e9-9b61-15b3a635c82f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:22:29.073371 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a9c874-e71f-43e9-9b61-15b3a635c82f None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:22:29.076983 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ef591c-b0ed-4280-afb2-fdae92010db3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfbb4d6809064f9cb9e06626ed7d6814', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '540a32ef88764e8c8d9201ae22f6ddaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:22:29.078151 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ef591c-b0ed-4280-afb2-fdae92010db3 service neutron] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:22:29.078457 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ef591c-b0ed-4280-afb2-fdae92010db3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:22:29.078726 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ef591c-b0ed-4280-afb2-fdae92010db3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:22:29.087611 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ef591c-b0ed-4280-afb2-fdae92010db3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:22:29.089072 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ef591c-b0ed-4280-afb2-fdae92010db3 service neutron] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:22:29.099115 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 762/1530] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 12:22:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 12:22:31.255294 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d98e56a-4dd3-4bd6-884d-f037682f7528 None None] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:22:31.255835 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d98e56a-4dd3-4bd6-884d-f037682f7528 None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:22:31.256094 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d98e56a-4dd3-4bd6-884d-f037682f7528 None None] PATH_INFO: `/` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:22:31.257585 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 769/1531] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 15 12:22:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:22:31.263214 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71e9b2b-1fc0-48f3-966a-2d50d994d92a None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:22:31.263673 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71e9b2b-1fc0-48f3-966a-2d50d994d92a None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:22:31.263783 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71e9b2b-1fc0-48f3-966a-2d50d994d92a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:22:31.284592 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-e71e9b2b-1fc0-48f3-966a-2d50d994d92a None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:22:31.285824 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-e71e9b2b-1fc0-48f3-966a-2d50d994d92a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:22:31.294165 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 763/1532] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 15 12:22:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:22:31.586747 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d98e56a-4dd3-4bd6-884d-f037682f7528 None None] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:22:31.588427 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-4d98e56a-4dd3-4bd6-884d-f037682f7528 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:22:31.602216 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d98e56a-4dd3-4bd6-884d-f037682f7528 None None] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:22:31.605733 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c0d3b1-1ca2-4d56-83ce-e9518d63cb23 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfbb4d6809064f9cb9e06626ed7d6814', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '540a32ef88764e8c8d9201ae22f6ddaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:22:31.606324 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c0d3b1-1ca2-4d56-83ce-e9518d63cb23 service neutron] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:22:31.606599 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c0d3b1-1ca2-4d56-83ce-e9518d63cb23 service neutron] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:22:31.606866 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c0d3b1-1ca2-4d56-83ce-e9518d63cb23 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:22:31.615737 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c0d3b1-1ca2-4d56-83ce-e9518d63cb23 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:22:31.617227 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c0d3b1-1ca2-4d56-83ce-e9518d63cb23 service neutron] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:22:31.626360 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 770/1533] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 12:22:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 12:22:35.585998 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24d52d2-479f-41d7-b5c8-bcf46fa581de None None] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:22:35.586585 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24d52d2-479f-41d7-b5c8-bcf46fa581de None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:22:35.586585 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24d52d2-479f-41d7-b5c8-bcf46fa581de None None] PATH_INFO: `/` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:22:35.587900 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 764/1534] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 15 12:22:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:22:35.593489 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19369f98-7efd-414f-9068-bf89449cc03b None None] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:22:35.593918 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19369f98-7efd-414f-9068-bf89449cc03b None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:22:35.594027 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19369f98-7efd-414f-9068-bf89449cc03b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:22:35.613246 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.auth.core [None req-19369f98-7efd-414f-9068-bf89449cc03b None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81449) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:22:35.614438 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-19369f98-7efd-414f-9068-bf89449cc03b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:22:35.622116 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 771/1535] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 15 12:22:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:22:35.931581 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24d52d2-479f-41d7-b5c8-bcf46fa581de None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:22:35.933260 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-e24d52d2-479f-41d7-b5c8-bcf46fa581de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:22:35.998630 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:22:36.012321 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24d52d2-479f-41d7-b5c8-bcf46fa581de None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:22:36.015868 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59c432e-a278-41c5-8ee6-747a21e12952 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8191d3ccb874529b32f8d0026db629d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '540a32ef88764e8c8d9201ae22f6ddaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:22:36.016469 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59c432e-a278-41c5-8ee6-747a21e12952 service cinder] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:22:36.016749 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59c432e-a278-41c5-8ee6-747a21e12952 service cinder] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:22:36.017045 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59c432e-a278-41c5-8ee6-747a21e12952 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:22:36.025724 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59c432e-a278-41c5-8ee6-747a21e12952 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:22:36.027128 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59c432e-a278-41c5-8ee6-747a21e12952 service cinder] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:22:36.036242 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 765/1536] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 12:22:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 12:22:38.161254 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8b1d54-c7cb-48e3-8c32-b294b972a4e9 None None] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:22:38.161753 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8b1d54-c7cb-48e3-8c32-b294b972a4e9 None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:22:38.161855 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8b1d54-c7cb-48e3-8c32-b294b972a4e9 None None] PATH_INFO: `/` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:22:38.163410 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 772/1537] 127.0.0.1 () {54 vars in 917 bytes} [Thu Sep 15 12:22:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:22:38.168871 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22bf884-d85b-4d5c-8d07-b2a469ae4f16 None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:22:38.169305 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22bf884-d85b-4d5c-8d07-b2a469ae4f16 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:22:38.169421 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22bf884-d85b-4d5c-8d07-b2a469ae4f16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:22:38.188975 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-c22bf884-d85b-4d5c-8d07-b2a469ae4f16 None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:22:38.190183 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-c22bf884-d85b-4d5c-8d07-b2a469ae4f16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:22:38.197956 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 766/1538] 127.0.0.1 () {58 vars in 1014 bytes} [Thu Sep 15 12:22:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:22:38.498708 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8b1d54-c7cb-48e3-8c32-b294b972a4e9 None None] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:22:38.500948 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-2b8b1d54-c7cb-48e3-8c32-b294b972a4e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:22:38.513844 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8b1d54-c7cb-48e3-8c32-b294b972a4e9 None None] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:22:38.517776 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56734b31-5967-4d06-9c29-f7e07aa0de82 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8191d3ccb874529b32f8d0026db629d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '540a32ef88764e8c8d9201ae22f6ddaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:22:38.518361 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56734b31-5967-4d06-9c29-f7e07aa0de82 service cinder] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:22:38.518636 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56734b31-5967-4d06-9c29-f7e07aa0de82 service cinder] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:22:38.518902 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56734b31-5967-4d06-9c29-f7e07aa0de82 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:22:38.527444 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56734b31-5967-4d06-9c29-f7e07aa0de82 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:22:38.528874 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56734b31-5967-4d06-9c29-f7e07aa0de82 service cinder] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:22:38.537871 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 773/1539] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 12:22:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 12:22:48.183612 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=11 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:22:48.533852 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=89 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:24:04.289507 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546ccab4-32b1-4643-8480-2e4edc0ffe3b None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:04.290155 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546ccab4-32b1-4643-8480-2e4edc0ffe3b None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:04.290155 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-546ccab4-32b1-4643-8480-2e4edc0ffe3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:04.316570 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-546ccab4-32b1-4643-8480-2e4edc0ffe3b None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:24:04.317846 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-546ccab4-32b1-4643-8480-2e4edc0ffe3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:24:04.330648 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 767/1540] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 12:24:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 12:24:04.339795 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d8379f-b88a-4cb7-bd26-ed5e59e90fc8 None None] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:04.340780 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d8379f-b88a-4cb7-bd26-ed5e59e90fc8 None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:04.341286 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d8379f-b88a-4cb7-bd26-ed5e59e90fc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:04.360263 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3b9b45-845b-46b4-97e8-8951a1a88b6c None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:04.360620 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3b9b45-845b-46b4-97e8-8951a1a88b6c None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:04.360840 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3b9b45-845b-46b4-97e8-8951a1a88b6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:04.373843 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.auth.core [None req-97d8379f-b88a-4cb7-bd26-ed5e59e90fc8 None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81449) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:24:04.375548 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-97d8379f-b88a-4cb7-bd26-ed5e59e90fc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:24:04.392299 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 774/1541] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 12:24:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:24:04.393796 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-bf3b9b45-845b-46b4-97e8-8951a1a88b6c None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:24:04.395346 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-bf3b9b45-845b-46b4-97e8-8951a1a88b6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:24:04.397511 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d8379f-b88a-4cb7-bd26-ed5e59e90fc8 None None] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:24:04.416543 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d8379f-b88a-4cb7-bd26-ed5e59e90fc8 None None] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:24:04.420959 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 768/1542] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 12:24:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 12:24:04.421750 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821d6e46-8445-4e1f-b562-2dab63594b12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:24:04.422398 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821d6e46-8445-4e1f-b562-2dab63594b12 admin admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:04.422734 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821d6e46-8445-4e1f-b562-2dab63594b12 admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:04.422949 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821d6e46-8445-4e1f-b562-2dab63594b12 admin admin] PATH_INFO: `/v3/domains` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:04.423152 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee5375c-127c-49db-8c8b-bd0762445089 None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:04.423500 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee5375c-127c-49db-8c8b-bd0762445089 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:04.423714 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821d6e46-8445-4e1f-b562-2dab63594b12 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81449) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:24:04.423817 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee5375c-127c-49db-8c8b-bd0762445089 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:04.424257 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821d6e46-8445-4e1f-b562-2dab63594b12 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:24:04.426797 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: WARNING py.warnings [None req-821d6e46-8445-4e1f-b562-2dab63594b12 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Sep 15 12:24:04.426797 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: warnings.warn(msg) Sep 15 12:24:04.426797 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: Sep 15 12:24:04.427718 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821d6e46-8445-4e1f-b562-2dab63594b12 admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:24:04.454645 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 775/1543] 10.222.0.15 () {58 vars in 1153 bytes} [Thu Sep 15 12:24:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:24:04.456016 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f8bafa-73e0-4cd1-a16e-d08f4fa43292 None None] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:04.456299 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f8bafa-73e0-4cd1-a16e-d08f4fa43292 None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:04.456569 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f8bafa-73e0-4cd1-a16e-d08f4fa43292 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:04.461820 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-7ee5375c-127c-49db-8c8b-bd0762445089 None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:24:04.463157 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-7ee5375c-127c-49db-8c8b-bd0762445089 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:24:04.480383 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.auth.core [None req-71f8bafa-73e0-4cd1-a16e-d08f4fa43292 None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81449) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:24:04.481664 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-71f8bafa-73e0-4cd1-a16e-d08f4fa43292 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:24:04.490519 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 769/1544] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 12:24:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:24:04.490748 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 776/1545] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 12:24:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:24:04.491285 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee5375c-127c-49db-8c8b-bd0762445089 None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:24:04.500381 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23853101-afaa-4699-b7ae-b290fe7a37b9 None None] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:04.500791 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23853101-afaa-4699-b7ae-b290fe7a37b9 None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:04.500982 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23853101-afaa-4699-b7ae-b290fe7a37b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:04.504438 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee5375c-127c-49db-8c8b-bd0762445089 None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:24:04.536001 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.auth.core [None req-23853101-afaa-4699-b7ae-b290fe7a37b9 None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81449) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:24:04.537248 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-23853101-afaa-4699-b7ae-b290fe7a37b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:24:04.542815 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71dda81-df98-43af-9205-3301432ac27e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:24:04.543397 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71dda81-df98-43af-9205-3301432ac27e admin admin] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:04.543677 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71dda81-df98-43af-9205-3301432ac27e admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:04.543947 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71dda81-df98-43af-9205-3301432ac27e admin admin] PATH_INFO: `/v3/projects` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:04.544645 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71dda81-df98-43af-9205-3301432ac27e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:24:04.545698 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: WARNING py.warnings [None req-a71dda81-df98-43af-9205-3301432ac27e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 15 12:24:04.545698 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: warnings.warn(msg) Sep 15 12:24:04.545698 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: Sep 15 12:24:04.546893 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71dda81-df98-43af-9205-3301432ac27e admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:24:04.564785 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 777/1546] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 12:24:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:24:04.565735 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23853101-afaa-4699-b7ae-b290fe7a37b9 None None] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:24:04.584875 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23853101-afaa-4699-b7ae-b290fe7a37b9 None None] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:24:04.601509 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca973b32-b7f5-44b7-87d1-d86b96b49818 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:24:04.602225 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca973b32-b7f5-44b7-87d1-d86b96b49818 admin admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:04.602524 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca973b32-b7f5-44b7-87d1-d86b96b49818 admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:04.602797 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca973b32-b7f5-44b7-87d1-d86b96b49818 admin admin] PATH_INFO: `/v3/domains` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:04.603900 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca973b32-b7f5-44b7-87d1-d86b96b49818 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81449) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:24:04.604113 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca973b32-b7f5-44b7-87d1-d86b96b49818 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:24:04.610366 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca973b32-b7f5-44b7-87d1-d86b96b49818 admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:24:04.638694 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 778/1547] 10.222.0.15 () {58 vars in 1153 bytes} [Thu Sep 15 12:24:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:24:04.640777 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 770/1548] 10.222.0.15 () {60 vars in 1153 bytes} [Thu Sep 15 12:24:04 2022] POST /identity/v3/projects => generated 375 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 12:24:04.645418 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca973b32-b7f5-44b7-87d1-d86b96b49818 admin admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:24:04.645691 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71dda81-df98-43af-9205-3301432ac27e admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:24:04.659639 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca973b32-b7f5-44b7-87d1-d86b96b49818 admin admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:24:04.766489 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71dda81-df98-43af-9205-3301432ac27e admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:24:04.769196 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60ccb9a-1cf5-4dcd-8602-754554a1f1e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:24:04.770995 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60ccb9a-1cf5-4dcd-8602-754554a1f1e2 admin admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:04.771275 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60ccb9a-1cf5-4dcd-8602-754554a1f1e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:04.771550 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60ccb9a-1cf5-4dcd-8602-754554a1f1e2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:04.772203 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60ccb9a-1cf5-4dcd-8602-754554a1f1e2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81449) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:24:04.772500 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60ccb9a-1cf5-4dcd-8602-754554a1f1e2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:24:04.774707 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60ccb9a-1cf5-4dcd-8602-754554a1f1e2 admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:24:04.779685 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0ba1ad-82a4-426a-8e78-6977a3dc8d7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:24:04.780295 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0ba1ad-82a4-426a-8e78-6977a3dc8d7f admin admin] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:04.780572 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0ba1ad-82a4-426a-8e78-6977a3dc8d7f admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:04.780870 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0ba1ad-82a4-426a-8e78-6977a3dc8d7f admin admin] PATH_INFO: `/v3/users` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:04.781583 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e0ba1ad-82a4-426a-8e78-6977a3dc8d7f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:24:04.783560 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e0ba1ad-82a4-426a-8e78-6977a3dc8d7f admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:24:04.811498 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 779/1549] 10.222.0.15 () {58 vars in 1153 bytes} [Thu Sep 15 12:24:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:24:04.813209 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60ccb9a-1cf5-4dcd-8602-754554a1f1e2 admin admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:24:04.828187 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60ccb9a-1cf5-4dcd-8602-754554a1f1e2 admin admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:24:04.831409 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 771/1550] 10.222.0.15 () {60 vars in 1144 bytes} [Thu Sep 15 12:24:04 2022] POST /identity/v3/users => generated 288 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 12:24:04.833013 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04fa5a3c-9cde-4de3-836e-0713297fb175 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:24:04.833453 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c56e20e-c5b4-4135-8898-dc951c2db413 None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:04.833570 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c56e20e-c5b4-4135-8898-dc951c2db413 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:04.833670 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fa5a3c-9cde-4de3-836e-0713297fb175 admin admin] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:04.833862 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c56e20e-c5b4-4135-8898-dc951c2db413 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:04.833978 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fa5a3c-9cde-4de3-836e-0713297fb175 admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:04.834214 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fa5a3c-9cde-4de3-836e-0713297fb175 admin admin] PATH_INFO: `/v3/projects` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:04.834907 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04fa5a3c-9cde-4de3-836e-0713297fb175 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:24:04.836828 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04fa5a3c-9cde-4de3-836e-0713297fb175 admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:24:04.854167 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-2c56e20e-c5b4-4135-8898-dc951c2db413 None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:24:04.855695 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-2c56e20e-c5b4-4135-8898-dc951c2db413 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:24:04.864489 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 772/1551] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 12:24:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:24:04.868278 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 780/1552] 10.222.0.15 () {60 vars in 1153 bytes} [Thu Sep 15 12:24:04 2022] POST /identity/v3/projects => generated 387 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 12:24:04.868543 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b2c03f-6c83-4ff2-b48c-74044cb3f1ce None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:04.868543 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b2c03f-6c83-4ff2-b48c-74044cb3f1ce None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:04.868543 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b2c03f-6c83-4ff2-b48c-74044cb3f1ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:04.868850 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a59361-023b-422f-9837-b48904a8e7c2 None None] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:04.868850 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a59361-023b-422f-9837-b48904a8e7c2 None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:04.868850 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a59361-023b-422f-9837-b48904a8e7c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:04.887622 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.auth.core [None req-12a59361-023b-422f-9837-b48904a8e7c2 None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81449) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:24:04.888866 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-12a59361-023b-422f-9837-b48904a8e7c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:24:04.889188 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-39b2c03f-6c83-4ff2-b48c-74044cb3f1ce None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:24:04.890339 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-39b2c03f-6c83-4ff2-b48c-74044cb3f1ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:24:04.947169 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 781/1553] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 12:24:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:24:04.947986 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a59361-023b-422f-9837-b48904a8e7c2 None None] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:24:04.957356 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 773/1554] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 12:24:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:24:04.957356 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a060f3f8-b8ee-4c12-aba0-647059e5eb8f None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:04.957356 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a060f3f8-b8ee-4c12-aba0-647059e5eb8f None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:04.957356 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a060f3f8-b8ee-4c12-aba0-647059e5eb8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:04.958499 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a59361-023b-422f-9837-b48904a8e7c2 None None] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:24:04.963323 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c26fe2-acc1-47b5-8312-7c4262a6b847 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:24:04.963969 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c26fe2-acc1-47b5-8312-7c4262a6b847 admin admin] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:04.964251 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c26fe2-acc1-47b5-8312-7c4262a6b847 admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:04.965401 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c26fe2-acc1-47b5-8312-7c4262a6b847 admin admin] PATH_INFO: `/v3/projects` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:04.965401 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c26fe2-acc1-47b5-8312-7c4262a6b847 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:24:04.967086 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c26fe2-acc1-47b5-8312-7c4262a6b847 admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:24:04.980424 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-a060f3f8-b8ee-4c12-aba0-647059e5eb8f None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:24:04.981680 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-a060f3f8-b8ee-4c12-aba0-647059e5eb8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:24:04.990422 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 774/1555] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 12:24:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:24:04.994167 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d9eb23-cc4b-4580-af21-2e1c49baee57 None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:04.994641 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d9eb23-cc4b-4580-af21-2e1c49baee57 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:04.995684 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d9eb23-cc4b-4580-af21-2e1c49baee57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:04.996380 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 782/1556] 10.222.0.15 () {60 vars in 1153 bytes} [Thu Sep 15 12:24:04 2022] POST /identity/v3/projects => generated 405 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 12:24:04.997236 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c26fe2-acc1-47b5-8312-7c4262a6b847 admin admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:24:05.004607 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:24:05.016518 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-90d9eb23-cc4b-4580-af21-2e1c49baee57 None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:24:05.017777 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-90d9eb23-cc4b-4580-af21-2e1c49baee57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:24:05.038548 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:24:05.062816 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c26fe2-acc1-47b5-8312-7c4262a6b847 admin admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:24:05.067563 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3c9461-dde8-41f7-9ffa-53111840d90c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:24:05.068500 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3c9461-dde8-41f7-9ffa-53111840d90c admin admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:05.068837 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3c9461-dde8-41f7-9ffa-53111840d90c admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:05.069075 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3c9461-dde8-41f7-9ffa-53111840d90c admin admin] PATH_INFO: `/v3/roles` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:05.069711 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3c9461-dde8-41f7-9ffa-53111840d90c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:24:05.071329 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3c9461-dde8-41f7-9ffa-53111840d90c admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:24:05.079589 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 775/1557] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 12:24:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:24:05.080444 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d9eb23-cc4b-4580-af21-2e1c49baee57 None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:24:05.084386 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 783/1558] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 12:24:04 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 12:24:05.084386 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a23d4cf-c374-4e13-9843-a84d39eac2b2 None None] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:05.084386 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a23d4cf-c374-4e13-9843-a84d39eac2b2 None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:05.084386 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a23d4cf-c374-4e13-9843-a84d39eac2b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:05.095179 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d9eb23-cc4b-4580-af21-2e1c49baee57 None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:24:05.100219 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3ffd98-e31f-451d-b795-0780f5901ba2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:24:05.100870 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3ffd98-e31f-451d-b795-0780f5901ba2 admin admin] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:05.101157 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3ffd98-e31f-451d-b795-0780f5901ba2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:05.101549 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3ffd98-e31f-451d-b795-0780f5901ba2 admin admin] PATH_INFO: `/v3/users` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:05.102190 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e3ffd98-e31f-451d-b795-0780f5901ba2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:24:05.104100 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e3ffd98-e31f-451d-b795-0780f5901ba2 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:24:05.105322 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.auth.core [None req-5a23d4cf-c374-4e13-9843-a84d39eac2b2 None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81449) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:24:05.106529 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-5a23d4cf-c374-4e13-9843-a84d39eac2b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:24:05.114535 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 784/1559] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 12:24:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:24:05.118301 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf7c6fa-e265-4b1a-96ee-e65918e1905f None None] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:05.118301 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf7c6fa-e265-4b1a-96ee-e65918e1905f None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:05.118301 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf7c6fa-e265-4b1a-96ee-e65918e1905f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:05.133986 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 776/1560] 10.222.0.15 () {60 vars in 1144 bytes} [Thu Sep 15 12:24:05 2022] POST /identity/v3/users => generated 294 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 12:24:05.133986 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11098c0-b4e3-4b1b-a111-1fbd604ff8a0 None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:05.133986 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11098c0-b4e3-4b1b-a111-1fbd604ff8a0 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:05.133986 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11098c0-b4e3-4b1b-a111-1fbd604ff8a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:05.138718 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.auth.core [None req-cdf7c6fa-e265-4b1a-96ee-e65918e1905f None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81449) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:24:05.139914 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-cdf7c6fa-e265-4b1a-96ee-e65918e1905f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:24:05.148915 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 785/1561] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 12:24:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:24:05.150535 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131cc68f-5287-411b-a8af-65c6fa761cb7 None None] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:05.150922 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131cc68f-5287-411b-a8af-65c6fa761cb7 None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:05.151303 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131cc68f-5287-411b-a8af-65c6fa761cb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:05.155613 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-c11098c0-b4e3-4b1b-a111-1fbd604ff8a0 None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:24:05.156810 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-c11098c0-b4e3-4b1b-a111-1fbd604ff8a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:24:05.164740 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 777/1562] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 12:24:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:24:05.166075 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11098c0-b4e3-4b1b-a111-1fbd604ff8a0 None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:24:05.171430 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.auth.core [None req-131cc68f-5287-411b-a8af-65c6fa761cb7 None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81449) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:24:05.172595 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-131cc68f-5287-411b-a8af-65c6fa761cb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:24:05.176027 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c11098c0-b4e3-4b1b-a111-1fbd604ff8a0 None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:24:05.180431 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 786/1563] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 12:24:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:24:05.180683 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e71198-771f-4200-9314-8e58d70229a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:24:05.181412 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e71198-771f-4200-9314-8e58d70229a3 admin admin] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:05.181811 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e71198-771f-4200-9314-8e58d70229a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:05.181930 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131cc68f-5287-411b-a8af-65c6fa761cb7 None None] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:24:05.182093 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e71198-771f-4200-9314-8e58d70229a3 admin admin] PATH_INFO: `/v3/users` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:05.183278 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e71198-771f-4200-9314-8e58d70229a3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:24:05.185248 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e71198-771f-4200-9314-8e58d70229a3 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:24:05.193505 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131cc68f-5287-411b-a8af-65c6fa761cb7 None None] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:24:05.198808 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d833ac-04e1-46b0-b002-c42a371e5455 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:24:05.199487 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d833ac-04e1-46b0-b002-c42a371e5455 admin admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:05.199789 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d833ac-04e1-46b0-b002-c42a371e5455 admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:05.200093 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d833ac-04e1-46b0-b002-c42a371e5455 admin admin] PATH_INFO: `/v3/roles` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:05.200743 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d833ac-04e1-46b0-b002-c42a371e5455 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:24:05.202613 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d833ac-04e1-46b0-b002-c42a371e5455 admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:24:05.207292 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 778/1564] 10.222.0.15 () {60 vars in 1144 bytes} [Thu Sep 15 12:24:05 2022] POST /identity/v3/users => generated 303 bytes in 42 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 12:24:05.208748 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3fe01c-114a-48c6-b8d8-761515dd5249 None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:05.209062 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3fe01c-114a-48c6-b8d8-761515dd5249 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:05.209338 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3fe01c-114a-48c6-b8d8-761515dd5249 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:05.214042 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 787/1565] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 12:24:05 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 12:24:05.215004 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d833ac-04e1-46b0-b002-c42a371e5455 admin admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:24:05.225672 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d833ac-04e1-46b0-b002-c42a371e5455 admin admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:24:05.229895 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-7f3fe01c-114a-48c6-b8d8-761515dd5249 None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:24:05.230799 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436feec0-5f44-4a32-a05d-9313712697e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:24:05.231082 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-7f3fe01c-114a-48c6-b8d8-761515dd5249 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:24:05.231435 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436feec0-5f44-4a32-a05d-9313712697e7 admin admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:05.231907 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436feec0-5f44-4a32-a05d-9313712697e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:05.232178 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436feec0-5f44-4a32-a05d-9313712697e7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:05.232852 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436feec0-5f44-4a32-a05d-9313712697e7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81449) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:24:05.233157 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436feec0-5f44-4a32-a05d-9313712697e7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:24:05.234790 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436feec0-5f44-4a32-a05d-9313712697e7 admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:24:05.239784 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 779/1566] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 12:24:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:24:05.240992 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3fe01c-114a-48c6-b8d8-761515dd5249 None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:24:05.255238 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 788/1567] 10.222.0.15 () {58 vars in 1153 bytes} [Thu Sep 15 12:24:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:24:05.255238 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436feec0-5f44-4a32-a05d-9313712697e7 admin admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:24:05.255522 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3fe01c-114a-48c6-b8d8-761515dd5249 None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:24:05.258693 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc36bbf1-fdcc-4039-a967-cd3a3f03c2b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:24:05.259310 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc36bbf1-fdcc-4039-a967-cd3a3f03c2b1 admin admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:05.259596 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc36bbf1-fdcc-4039-a967-cd3a3f03c2b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:05.259876 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc36bbf1-fdcc-4039-a967-cd3a3f03c2b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:05.260505 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436feec0-5f44-4a32-a05d-9313712697e7 admin admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:24:05.260636 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc36bbf1-fdcc-4039-a967-cd3a3f03c2b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:24:05.262188 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc36bbf1-fdcc-4039-a967-cd3a3f03c2b1 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:24:05.265133 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be9b5f4-b3f9-4b92-ad40-f98094af269a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:24:05.265717 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be9b5f4-b3f9-4b92-ad40-f98094af269a admin admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:05.265996 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be9b5f4-b3f9-4b92-ad40-f98094af269a admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:05.266270 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be9b5f4-b3f9-4b92-ad40-f98094af269a admin admin] PATH_INFO: `/v3/domains` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:05.266891 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be9b5f4-b3f9-4b92-ad40-f98094af269a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81449) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:24:05.267188 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be9b5f4-b3f9-4b92-ad40-f98094af269a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:24:05.268743 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be9b5f4-b3f9-4b92-ad40-f98094af269a admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:24:05.272248 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 780/1568] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 12:24:05 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 12:24:05.273064 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc36bbf1-fdcc-4039-a967-cd3a3f03c2b1 admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:24:05.284909 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 789/1569] 10.222.0.15 () {58 vars in 1153 bytes} [Thu Sep 15 12:24:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:24:05.284909 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be9b5f4-b3f9-4b92-ad40-f98094af269a admin admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:24:05.287378 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc36bbf1-fdcc-4039-a967-cd3a3f03c2b1 admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:24:05.290596 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be9b5f4-b3f9-4b92-ad40-f98094af269a admin admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:24:05.292701 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0854315-8cf5-4fa8-9661-78a7be4dda25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:24:05.293352 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0854315-8cf5-4fa8-9661-78a7be4dda25 admin admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:05.293633 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0854315-8cf5-4fa8-9661-78a7be4dda25 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:05.293905 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0854315-8cf5-4fa8-9661-78a7be4dda25 admin admin] PATH_INFO: `/v3/domains` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:05.294544 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0854315-8cf5-4fa8-9661-78a7be4dda25 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:24:05.294848 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0854315-8cf5-4fa8-9661-78a7be4dda25 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:24:05.295600 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1513f953-aa3a-47a0-b4b4-6239f8f28ccb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:24:05.296174 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1513f953-aa3a-47a0-b4b4-6239f8f28ccb admin admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:05.296422 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0854315-8cf5-4fa8-9661-78a7be4dda25 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:24:05.296542 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1513f953-aa3a-47a0-b4b4-6239f8f28ccb admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:05.296731 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1513f953-aa3a-47a0-b4b4-6239f8f28ccb admin admin] PATH_INFO: `/v3/domains` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:05.297372 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1513f953-aa3a-47a0-b4b4-6239f8f28ccb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81449) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:24:05.297669 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1513f953-aa3a-47a0-b4b4-6239f8f28ccb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:24:05.299232 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1513f953-aa3a-47a0-b4b4-6239f8f28ccb admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:24:05.308466 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 781/1570] 10.222.0.15 () {58 vars in 1153 bytes} [Thu Sep 15 12:24:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:24:05.317629 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 790/1571] 10.222.0.15 () {58 vars in 1153 bytes} [Thu Sep 15 12:24:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:24:05.317629 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1513f953-aa3a-47a0-b4b4-6239f8f28ccb admin admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:24:05.317887 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0854315-8cf5-4fa8-9661-78a7be4dda25 admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:24:05.326361 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0854315-8cf5-4fa8-9661-78a7be4dda25 admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:24:05.327455 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1513f953-aa3a-47a0-b4b4-6239f8f28ccb admin admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:24:05.330895 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e50b8b-3725-45ec-ba89-a570fc4bba7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:24:05.331502 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e50b8b-3725-45ec-ba89-a570fc4bba7d admin admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:05.331783 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e50b8b-3725-45ec-ba89-a570fc4bba7d admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:05.331899 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c652678-3fe7-423c-9a70-c670fd3a82db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:24:05.332052 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e50b8b-3725-45ec-ba89-a570fc4bba7d admin admin] PATH_INFO: `/v3/roles` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:05.332519 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c652678-3fe7-423c-9a70-c670fd3a82db admin admin] REQUEST_METHOD: `PUT` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:05.332685 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e50b8b-3725-45ec-ba89-a570fc4bba7d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:24:05.332803 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c652678-3fe7-423c-9a70-c670fd3a82db admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:05.333079 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c652678-3fe7-423c-9a70-c670fd3a82db admin admin] PATH_INFO: `/v3/projects/705a7022acf04881bed0e40d7a491dd4/users/b5c77cfb2ad8404f9d0dff4f9bdb98ba/roles/0679d6628a25438d90dc4bdc0401eda2` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:05.334304 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e50b8b-3725-45ec-ba89-a570fc4bba7d admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:24:05.344408 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 782/1572] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 12:24:05 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 12:24:05.345237 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e50b8b-3725-45ec-ba89-a570fc4bba7d admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:24:05.348116 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c652678-3fe7-423c-9a70-c670fd3a82db admin admin] RBAC: Authorizing `identity:create_grant(project_id=705a7022acf04881bed0e40d7a491dd4, user_id=b5c77cfb2ad8404f9d0dff4f9bdb98ba, role_id=0679d6628a25438d90dc4bdc0401eda2)` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:24:05.350795 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c652678-3fe7-423c-9a70-c670fd3a82db admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:24:05.355258 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e50b8b-3725-45ec-ba89-a570fc4bba7d admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:24:05.360138 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a3db50-e704-4e0f-a37d-f2b0175a60cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:24:05.360723 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a3db50-e704-4e0f-a37d-f2b0175a60cd admin admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:05.361043 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a3db50-e704-4e0f-a37d-f2b0175a60cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:05.361317 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a3db50-e704-4e0f-a37d-f2b0175a60cd admin admin] PATH_INFO: `/v3/domains` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:05.361951 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a3db50-e704-4e0f-a37d-f2b0175a60cd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:24:05.362255 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a3db50-e704-4e0f-a37d-f2b0175a60cd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:24:05.363823 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a3db50-e704-4e0f-a37d-f2b0175a60cd admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:24:05.365367 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 791/1573] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 15 12:24:05 2022] PUT /identity/v3/projects/705a7022acf04881bed0e40d7a491dd4/users/b5c77cfb2ad8404f9d0dff4f9bdb98ba/roles/0679d6628a25438d90dc4bdc0401eda2 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:24:05.366442 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c652678-3fe7-423c-9a70-c670fd3a82db admin admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:24:05.375853 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 783/1574] 10.222.0.15 () {58 vars in 1153 bytes} [Thu Sep 15 12:24:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:24:05.376657 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a3db50-e704-4e0f-a37d-f2b0175a60cd admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:24:05.386412 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a3db50-e704-4e0f-a37d-f2b0175a60cd admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:24:05.432249 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c652678-3fe7-423c-9a70-c670fd3a82db admin admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:24:05.436959 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edaf29e4-67ce-4e98-b546-ddb5ebd2c71a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:24:05.437566 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edaf29e4-67ce-4e98-b546-ddb5ebd2c71a admin admin] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:05.437855 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edaf29e4-67ce-4e98-b546-ddb5ebd2c71a admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:05.438118 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edaf29e4-67ce-4e98-b546-ddb5ebd2c71a admin admin] PATH_INFO: `/v3/projects` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:05.438738 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edaf29e4-67ce-4e98-b546-ddb5ebd2c71a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:24:05.440697 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edaf29e4-67ce-4e98-b546-ddb5ebd2c71a admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:24:05.443083 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356cdedd-e1a9-4267-9f7f-bf915c681dfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:24:05.443695 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356cdedd-e1a9-4267-9f7f-bf915c681dfd admin admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:05.443973 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356cdedd-e1a9-4267-9f7f-bf915c681dfd admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:05.444275 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356cdedd-e1a9-4267-9f7f-bf915c681dfd admin admin] PATH_INFO: `/v3/roles` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:05.444954 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356cdedd-e1a9-4267-9f7f-bf915c681dfd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:24:05.446526 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356cdedd-e1a9-4267-9f7f-bf915c681dfd admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:24:05.452138 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:24:05.457527 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 784/1575] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 12:24:05 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 12:24:05.458416 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356cdedd-e1a9-4267-9f7f-bf915c681dfd admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:24:05.469429 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356cdedd-e1a9-4267-9f7f-bf915c681dfd admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:24:05.473963 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 792/1576] 10.222.0.15 () {60 vars in 1153 bytes} [Thu Sep 15 12:24:05 2022] POST /identity/v3/projects => generated 383 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 12:24:05.473963 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edaf29e4-67ce-4e98-b546-ddb5ebd2c71a admin admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:24:05.496923 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:24:05.532184 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad12af0-c0c0-415b-b4d4-486f2c746259 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:24:05.533018 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad12af0-c0c0-415b-b4d4-486f2c746259 admin admin] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:05.533299 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad12af0-c0c0-415b-b4d4-486f2c746259 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:05.533573 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad12af0-c0c0-415b-b4d4-486f2c746259 admin admin] PATH_INFO: `/v3/projects` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:05.534194 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad12af0-c0c0-415b-b4d4-486f2c746259 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:24:05.535858 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edaf29e4-67ce-4e98-b546-ddb5ebd2c71a admin admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:24:05.536082 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad12af0-c0c0-415b-b4d4-486f2c746259 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:24:05.544629 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd35dbac-dcb1-4956-968f-49b8bbd668a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:24:05.545307 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd35dbac-dcb1-4956-968f-49b8bbd668a6 admin admin] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:05.545588 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd35dbac-dcb1-4956-968f-49b8bbd668a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:05.545864 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd35dbac-dcb1-4956-968f-49b8bbd668a6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:05.546499 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd35dbac-dcb1-4956-968f-49b8bbd668a6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:24:05.548406 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd35dbac-dcb1-4956-968f-49b8bbd668a6 admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:24:05.567312 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 785/1577] 10.222.0.15 () {60 vars in 1153 bytes} [Thu Sep 15 12:24:05 2022] POST /identity/v3/projects => generated 383 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 12:24:05.567312 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad12af0-c0c0-415b-b4d4-486f2c746259 admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:24:05.580682 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 793/1578] 10.222.0.15 () {60 vars in 1153 bytes} [Thu Sep 15 12:24:05 2022] POST /identity/v3/projects => generated 375 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 12:24:05.580682 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd35dbac-dcb1-4956-968f-49b8bbd668a6 admin admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:24:05.626551 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad12af0-c0c0-415b-b4d4-486f2c746259 admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:24:05.639922 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd35dbac-dcb1-4956-968f-49b8bbd668a6 admin admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:24:05.645132 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103e013b-ff47-43ef-b213-22359fad9174 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:24:05.645752 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103e013b-ff47-43ef-b213-22359fad9174 admin admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:05.646031 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103e013b-ff47-43ef-b213-22359fad9174 admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:05.646306 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103e013b-ff47-43ef-b213-22359fad9174 admin admin] PATH_INFO: `/v3/roles` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:05.646960 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103e013b-ff47-43ef-b213-22359fad9174 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:24:05.648558 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103e013b-ff47-43ef-b213-22359fad9174 admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:24:05.658625 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 794/1579] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 12:24:05 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 12:24:05.659801 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbd9295-de62-44e4-805b-c04605837eca None None] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:05.660084 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbd9295-de62-44e4-805b-c04605837eca None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:05.660359 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbd9295-de62-44e4-805b-c04605837eca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:05.678497 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.auth.core [None req-3cbd9295-de62-44e4-805b-c04605837eca None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81449) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:24:05.679697 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-3cbd9295-de62-44e4-805b-c04605837eca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:24:05.683815 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702e1553-2d9c-4ba1-9787-9226db1739ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:24:05.684433 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702e1553-2d9c-4ba1-9787-9226db1739ea admin admin] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:05.684709 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702e1553-2d9c-4ba1-9787-9226db1739ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:05.685029 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702e1553-2d9c-4ba1-9787-9226db1739ea admin admin] PATH_INFO: `/v3/projects` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:05.685648 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-702e1553-2d9c-4ba1-9787-9226db1739ea admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:24:05.687528 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-702e1553-2d9c-4ba1-9787-9226db1739ea admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:24:05.687998 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 795/1580] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 12:24:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:24:05.688742 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbd9295-de62-44e4-805b-c04605837eca None None] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:24:05.699523 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbd9295-de62-44e4-805b-c04605837eca None None] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:24:05.704125 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a2ed90-707d-4062-835e-8188ea497eee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:24:05.704777 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a2ed90-707d-4062-835e-8188ea497eee admin admin] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:05.705051 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a2ed90-707d-4062-835e-8188ea497eee admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:05.705329 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a2ed90-707d-4062-835e-8188ea497eee admin admin] PATH_INFO: `/v3/projects` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:05.705988 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a2ed90-707d-4062-835e-8188ea497eee admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:24:05.708026 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a2ed90-707d-4062-835e-8188ea497eee admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:24:05.718219 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 786/1581] 10.222.0.15 () {60 vars in 1153 bytes} [Thu Sep 15 12:24:05 2022] POST /identity/v3/projects => generated 365 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 12:24:05.718219 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702e1553-2d9c-4ba1-9787-9226db1739ea admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:24:05.739464 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 796/1582] 10.222.0.15 () {60 vars in 1153 bytes} [Thu Sep 15 12:24:05 2022] POST /identity/v3/projects => generated 371 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 12:24:05.739464 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a2ed90-707d-4062-835e-8188ea497eee admin admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:24:05.765424 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:24:05.782164 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-702e1553-2d9c-4ba1-9787-9226db1739ea admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:24:05.783999 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:24:05.800895 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a2ed90-707d-4062-835e-8188ea497eee admin admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:24:05.805537 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76d207f-d2f6-4381-baac-467cbe238a7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:24:05.806143 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76d207f-d2f6-4381-baac-467cbe238a7f admin admin] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:05.806427 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76d207f-d2f6-4381-baac-467cbe238a7f admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:05.806703 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76d207f-d2f6-4381-baac-467cbe238a7f admin admin] PATH_INFO: `/v3/users` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:05.807333 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76d207f-d2f6-4381-baac-467cbe238a7f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:24:05.808932 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: WARNING py.warnings [None req-d76d207f-d2f6-4381-baac-467cbe238a7f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 15 12:24:05.808932 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: warnings.warn(msg) Sep 15 12:24:05.808932 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: Sep 15 12:24:05.810141 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76d207f-d2f6-4381-baac-467cbe238a7f admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:24:05.835402 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 797/1583] 10.222.0.15 () {60 vars in 1144 bytes} [Thu Sep 15 12:24:05 2022] POST /identity/v3/users => generated 292 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 12:24:05.835402 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76d207f-d2f6-4381-baac-467cbe238a7f admin admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:24:05.842374 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76d207f-d2f6-4381-baac-467cbe238a7f admin admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:24:05.843632 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ed91c5-7e46-4b30-a1ad-ae5ce0f6bdb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:24:05.844321 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ed91c5-7e46-4b30-a1ad-ae5ce0f6bdb5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:05.844635 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ed91c5-7e46-4b30-a1ad-ae5ce0f6bdb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:05.844897 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ed91c5-7e46-4b30-a1ad-ae5ce0f6bdb5 admin admin] PATH_INFO: `/v3/projects/8d77a46e3c75435ca3632e12a4cb642e/users/1c42d5435a514bc0880b69fbbc163d32/roles/0679d6628a25438d90dc4bdc0401eda2` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:05.847234 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1937cb9-cdc5-4dca-8884-d911982cd52d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:24:05.847814 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1937cb9-cdc5-4dca-8884-d911982cd52d admin admin] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:05.848090 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1937cb9-cdc5-4dca-8884-d911982cd52d admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:05.848362 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1937cb9-cdc5-4dca-8884-d911982cd52d admin admin] PATH_INFO: `/v3/users` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:05.848998 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1937cb9-cdc5-4dca-8884-d911982cd52d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:24:05.850901 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1937cb9-cdc5-4dca-8884-d911982cd52d admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:24:05.860744 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ed91c5-7e46-4b30-a1ad-ae5ce0f6bdb5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8d77a46e3c75435ca3632e12a4cb642e, user_id=1c42d5435a514bc0880b69fbbc163d32, role_id=0679d6628a25438d90dc4bdc0401eda2)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:24:05.862162 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: WARNING py.warnings [None req-22ed91c5-7e46-4b30-a1ad-ae5ce0f6bdb5 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 15 12:24:05.862162 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: warnings.warn(msg) Sep 15 12:24:05.862162 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: Sep 15 12:24:05.864389 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ed91c5-7e46-4b30-a1ad-ae5ce0f6bdb5 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:24:05.872087 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 798/1584] 10.222.0.15 () {60 vars in 1144 bytes} [Thu Sep 15 12:24:05 2022] POST /identity/v3/users => generated 292 bytes in 39 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 12:24:05.872893 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1937cb9-cdc5-4dca-8884-d911982cd52d admin admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:24:05.878679 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 787/1585] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 15 12:24:05 2022] PUT /identity/v3/projects/8d77a46e3c75435ca3632e12a4cb642e/users/1c42d5435a514bc0880b69fbbc163d32/roles/0679d6628a25438d90dc4bdc0401eda2 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:24:05.879516 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ed91c5-7e46-4b30-a1ad-ae5ce0f6bdb5 admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:24:05.882648 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1937cb9-cdc5-4dca-8884-d911982cd52d admin admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:24:05.938778 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428c8cb2-c149-47f0-8154-caa2fd318e1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:24:05.939550 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428c8cb2-c149-47f0-8154-caa2fd318e1f admin admin] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:05.939931 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428c8cb2-c149-47f0-8154-caa2fd318e1f admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:05.940304 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428c8cb2-c149-47f0-8154-caa2fd318e1f admin admin] PATH_INFO: `/v3/users` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:05.940428 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ed91c5-7e46-4b30-a1ad-ae5ce0f6bdb5 admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:24:05.941168 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-428c8cb2-c149-47f0-8154-caa2fd318e1f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:24:05.943085 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-428c8cb2-c149-47f0-8154-caa2fd318e1f admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:24:05.944954 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640b7933-ef75-45d4-a19e-36255618e1d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:24:05.945617 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640b7933-ef75-45d4-a19e-36255618e1d2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:05.945898 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640b7933-ef75-45d4-a19e-36255618e1d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:05.946169 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640b7933-ef75-45d4-a19e-36255618e1d2 admin admin] PATH_INFO: `/v3/projects/3191251608ff40dd94189a51a45adf76/users/d4ea50ffefa34b17808f3458fb0654d6/roles/0679d6628a25438d90dc4bdc0401eda2` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:05.963267 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640b7933-ef75-45d4-a19e-36255618e1d2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3191251608ff40dd94189a51a45adf76, user_id=d4ea50ffefa34b17808f3458fb0654d6, role_id=0679d6628a25438d90dc4bdc0401eda2)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:24:05.966060 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640b7933-ef75-45d4-a19e-36255618e1d2 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:24:05.968586 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 799/1586] 10.222.0.15 () {60 vars in 1144 bytes} [Thu Sep 15 12:24:05 2022] POST /identity/v3/users => generated 288 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 12:24:05.969605 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428c8cb2-c149-47f0-8154-caa2fd318e1f admin admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:24:05.988288 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 788/1587] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 15 12:24:05 2022] PUT /identity/v3/projects/3191251608ff40dd94189a51a45adf76/users/d4ea50ffefa34b17808f3458fb0654d6/roles/0679d6628a25438d90dc4bdc0401eda2 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:24:05.989417 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640b7933-ef75-45d4-a19e-36255618e1d2 admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:24:06.036730 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428c8cb2-c149-47f0-8154-caa2fd318e1f admin admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:24:06.050748 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:24:06.054415 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640b7933-ef75-45d4-a19e-36255618e1d2 admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:24:06.060017 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e635fc96-1cff-4d93-8dd7-8f8463de0f4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:24:06.060622 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e635fc96-1cff-4d93-8dd7-8f8463de0f4c admin admin] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:06.060931 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e635fc96-1cff-4d93-8dd7-8f8463de0f4c admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:06.061201 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e635fc96-1cff-4d93-8dd7-8f8463de0f4c admin admin] PATH_INFO: `/v3/users` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:06.061827 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e635fc96-1cff-4d93-8dd7-8f8463de0f4c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:24:06.063711 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e635fc96-1cff-4d93-8dd7-8f8463de0f4c admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:24:06.085871 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 789/1588] 10.222.0.15 () {60 vars in 1144 bytes} [Thu Sep 15 12:24:05 2022] POST /identity/v3/users => generated 283 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 12:24:06.086552 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e635fc96-1cff-4d93-8dd7-8f8463de0f4c admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:24:06.089424 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbbcf9b-ec04-4e61-9529-81c448f246d8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfbb4d6809064f9cb9e06626ed7d6814', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '540a32ef88764e8c8d9201ae22f6ddaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:24:06.090032 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbbcf9b-ec04-4e61-9529-81c448f246d8 service neutron] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:06.090304 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbbcf9b-ec04-4e61-9529-81c448f246d8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:06.090568 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbbcf9b-ec04-4e61-9529-81c448f246d8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:06.100495 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cbbcf9b-ec04-4e61-9529-81c448f246d8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:24:06.100656 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e635fc96-1cff-4d93-8dd7-8f8463de0f4c admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:24:06.101947 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cbbcf9b-ec04-4e61-9529-81c448f246d8 service neutron] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:24:06.105857 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fceb86a3-44fb-4df8-b957-109c563eafad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:24:06.106631 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fceb86a3-44fb-4df8-b957-109c563eafad admin admin] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:06.106911 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fceb86a3-44fb-4df8-b957-109c563eafad admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:06.107708 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fceb86a3-44fb-4df8-b957-109c563eafad admin admin] PATH_INFO: `/v3/users` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:06.108338 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fceb86a3-44fb-4df8-b957-109c563eafad admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:24:06.110256 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fceb86a3-44fb-4df8-b957-109c563eafad admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:24:06.111997 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 800/1589] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 12:24:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 12:24:06.112912 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbbcf9b-ec04-4e61-9529-81c448f246d8 service neutron] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:24:06.119788 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=91,keystone:INSERT=9 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:24:06.124543 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbbcf9b-ec04-4e61-9529-81c448f246d8 service neutron] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:24:06.131834 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef11748-e555-48ad-869e-2754e15a977c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:24:06.132688 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef11748-e555-48ad-869e-2754e15a977c admin admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:06.133176 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef11748-e555-48ad-869e-2754e15a977c admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:06.133307 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef11748-e555-48ad-869e-2754e15a977c admin admin] PATH_INFO: `/v3/roles` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:06.133962 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef11748-e555-48ad-869e-2754e15a977c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:24:06.135612 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef11748-e555-48ad-869e-2754e15a977c admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:24:06.140053 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 790/1590] 10.222.0.15 () {60 vars in 1144 bytes} [Thu Sep 15 12:24:06 2022] POST /identity/v3/users => generated 286 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 12:24:06.140053 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fceb86a3-44fb-4df8-b957-109c563eafad admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:24:06.147132 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 801/1591] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 12:24:06 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 12:24:06.149541 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fceb86a3-44fb-4df8-b957-109c563eafad admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:24:06.151276 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04708bdc-1c9b-4b71-8ee8-c84d0f5fcd86 None None] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:06.151558 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04708bdc-1c9b-4b71-8ee8-c84d0f5fcd86 None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:06.151831 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04708bdc-1c9b-4b71-8ee8-c84d0f5fcd86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:06.155093 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c417ba13-ce62-4f39-b00b-eb579ad03228 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:24:06.155723 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c417ba13-ce62-4f39-b00b-eb579ad03228 admin admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:06.156004 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c417ba13-ce62-4f39-b00b-eb579ad03228 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:06.156271 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c417ba13-ce62-4f39-b00b-eb579ad03228 admin admin] PATH_INFO: `/v3/roles` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:06.157013 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c417ba13-ce62-4f39-b00b-eb579ad03228 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:24:06.158759 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c417ba13-ce62-4f39-b00b-eb579ad03228 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:24:06.173282 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 791/1592] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 12:24:06 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 12:24:06.173487 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.auth.core [None req-04708bdc-1c9b-4b71-8ee8-c84d0f5fcd86 None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81449) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:24:06.173593 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c417ba13-ce62-4f39-b00b-eb579ad03228 admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:24:06.173693 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-04708bdc-1c9b-4b71-8ee8-c84d0f5fcd86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:24:06.181038 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c417ba13-ce62-4f39-b00b-eb579ad03228 admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:24:06.181206 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 802/1593] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 12:24:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:24:06.182214 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cea610-2ad3-4627-b26d-b2e61b7a42ef None None] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:06.183153 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cea610-2ad3-4627-b26d-b2e61b7a42ef None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:06.183424 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cea610-2ad3-4627-b26d-b2e61b7a42ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:06.188910 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd1a4f8-9ed6-4c33-affb-beb5e1f1c53b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:24:06.189716 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd1a4f8-9ed6-4c33-affb-beb5e1f1c53b admin admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:06.190632 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd1a4f8-9ed6-4c33-affb-beb5e1f1c53b admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:06.190914 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd1a4f8-9ed6-4c33-affb-beb5e1f1c53b admin admin] PATH_INFO: `/v3/roles` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:06.191569 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd1a4f8-9ed6-4c33-affb-beb5e1f1c53b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:24:06.193606 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd1a4f8-9ed6-4c33-affb-beb5e1f1c53b admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:24:06.204194 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 792/1594] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 12:24:06 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 12:24:06.206446 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd1a4f8-9ed6-4c33-affb-beb5e1f1c53b admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:24:06.216692 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.auth.core [None req-44cea610-2ad3-4627-b26d-b2e61b7a42ef None None] MFA Rules not processed for user `d4ea50ffefa34b17808f3458fb0654d6`. Rule list: `[]` (Enabled: `True`). {{(pid=81449) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:24:06.218464 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-44cea610-2ad3-4627-b26d-b2e61b7a42ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:24:06.218636 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd1a4f8-9ed6-4c33-affb-beb5e1f1c53b admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:24:06.224781 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c1d5f9-12bd-4884-a96e-9ff4666f0276 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:24:06.225542 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c1d5f9-12bd-4884-a96e-9ff4666f0276 admin admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:06.225854 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c1d5f9-12bd-4884-a96e-9ff4666f0276 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:06.226072 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c1d5f9-12bd-4884-a96e-9ff4666f0276 admin admin] PATH_INFO: `/v3/roles` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:06.226716 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c1d5f9-12bd-4884-a96e-9ff4666f0276 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:24:06.228349 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c1d5f9-12bd-4884-a96e-9ff4666f0276 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:24:06.240953 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 793/1595] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 12:24:06 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 12:24:06.242205 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c1d5f9-12bd-4884-a96e-9ff4666f0276 admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:24:06.253197 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c1d5f9-12bd-4884-a96e-9ff4666f0276 admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:24:06.257984 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99ddc40-f924-4480-8edd-7780df84dca9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:24:06.258687 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99ddc40-f924-4480-8edd-7780df84dca9 admin admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:06.258974 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99ddc40-f924-4480-8edd-7780df84dca9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:06.259254 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99ddc40-f924-4480-8edd-7780df84dca9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:06.259981 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99ddc40-f924-4480-8edd-7780df84dca9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:24:06.262257 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99ddc40-f924-4480-8edd-7780df84dca9 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:24:06.275489 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 794/1596] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 12:24:06 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 12:24:06.276504 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99ddc40-f924-4480-8edd-7780df84dca9 admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:24:06.288078 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99ddc40-f924-4480-8edd-7780df84dca9 admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:24:06.293553 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35ce673-7ca5-4a84-8ee7-7470c36544ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:24:06.294268 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35ce673-7ca5-4a84-8ee7-7470c36544ed admin admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:06.294730 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35ce673-7ca5-4a84-8ee7-7470c36544ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:06.295100 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35ce673-7ca5-4a84-8ee7-7470c36544ed admin admin] PATH_INFO: `/v3/roles` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:06.295834 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35ce673-7ca5-4a84-8ee7-7470c36544ed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:24:06.297413 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 803/1597] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 12:24:06 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:24:06.301203 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35ce673-7ca5-4a84-8ee7-7470c36544ed admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:24:06.301508 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44cea610-2ad3-4627-b26d-b2e61b7a42ef None None] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:24:06.308910 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 795/1598] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 12:24:06 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 12:24:06.309785 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35ce673-7ca5-4a84-8ee7-7470c36544ed admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:24:06.316782 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44cea610-2ad3-4627-b26d-b2e61b7a42ef None None] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:24:06.320993 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35ce673-7ca5-4a84-8ee7-7470c36544ed admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:24:06.321478 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd053489-c92f-41cf-9bf6-fe2a48107c0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:24:06.322180 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd053489-c92f-41cf-9bf6-fe2a48107c0a admin admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:06.322535 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd053489-c92f-41cf-9bf6-fe2a48107c0a admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:06.322858 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd053489-c92f-41cf-9bf6-fe2a48107c0a admin admin] PATH_INFO: `/v3/roles` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:06.324094 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd053489-c92f-41cf-9bf6-fe2a48107c0a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:24:06.325792 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd053489-c92f-41cf-9bf6-fe2a48107c0a admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:24:06.326530 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f67c63-bb87-48d2-a3ab-303419984771 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfbb4d6809064f9cb9e06626ed7d6814', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '540a32ef88764e8c8d9201ae22f6ddaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:24:06.327311 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f67c63-bb87-48d2-a3ab-303419984771 service neutron] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:06.327787 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f67c63-bb87-48d2-a3ab-303419984771 service neutron] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:06.328073 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f67c63-bb87-48d2-a3ab-303419984771 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:06.339722 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 804/1599] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 12:24:06 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 12:24:06.339722 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd053489-c92f-41cf-9bf6-fe2a48107c0a admin admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:24:06.341327 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f67c63-bb87-48d2-a3ab-303419984771 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:24:06.342859 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4f67c63-bb87-48d2-a3ab-303419984771 service neutron] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:24:06.347652 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd053489-c92f-41cf-9bf6-fe2a48107c0a admin admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:24:06.352700 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82f15c2-59ba-4e1b-adff-0f444612bccc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:24:06.353293 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82f15c2-59ba-4e1b-adff-0f444612bccc admin admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:06.353618 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82f15c2-59ba-4e1b-adff-0f444612bccc admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:06.353804 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82f15c2-59ba-4e1b-adff-0f444612bccc admin admin] PATH_INFO: `/v3/roles` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:06.354458 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f82f15c2-59ba-4e1b-adff-0f444612bccc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:24:06.356057 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f82f15c2-59ba-4e1b-adff-0f444612bccc admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:24:06.362813 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 796/1600] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 12:24:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 12:24:06.364137 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f67c63-bb87-48d2-a3ab-303419984771 service neutron] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:24:06.366719 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 805/1601] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 12:24:06 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 12:24:06.367672 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82f15c2-59ba-4e1b-adff-0f444612bccc admin admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:24:06.378787 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82f15c2-59ba-4e1b-adff-0f444612bccc admin admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:24:06.380610 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f67c63-bb87-48d2-a3ab-303419984771 service neutron] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:24:06.384051 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d059d8a5-e01b-4e30-9d33-345f1f729546 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:24:06.384835 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d059d8a5-e01b-4e30-9d33-345f1f729546 admin admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:06.385215 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d059d8a5-e01b-4e30-9d33-345f1f729546 admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:06.385636 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d059d8a5-e01b-4e30-9d33-345f1f729546 admin admin] PATH_INFO: `/v3/roles` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:06.386944 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d059d8a5-e01b-4e30-9d33-345f1f729546 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:24:06.387502 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba16dc8e-8ff5-4b5f-b285-e4032ffd76f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:24:06.388523 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba16dc8e-8ff5-4b5f-b285-e4032ffd76f2 admin admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:06.388639 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba16dc8e-8ff5-4b5f-b285-e4032ffd76f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:06.388820 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d059d8a5-e01b-4e30-9d33-345f1f729546 admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:24:06.389057 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba16dc8e-8ff5-4b5f-b285-e4032ffd76f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:06.389790 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba16dc8e-8ff5-4b5f-b285-e4032ffd76f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:24:06.392132 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba16dc8e-8ff5-4b5f-b285-e4032ffd76f2 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:24:06.400840 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 806/1602] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 12:24:06 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 12:24:06.402057 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d059d8a5-e01b-4e30-9d33-345f1f729546 admin admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:24:06.410842 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 797/1603] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 12:24:06 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 12:24:06.411734 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba16dc8e-8ff5-4b5f-b285-e4032ffd76f2 admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:24:06.416408 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d059d8a5-e01b-4e30-9d33-345f1f729546 admin admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:24:06.422025 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ef2478-586e-4c32-8398-fd0a0d44666c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:24:06.422787 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ef2478-586e-4c32-8398-fd0a0d44666c admin admin] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:06.423133 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ef2478-586e-4c32-8398-fd0a0d44666c admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:06.423379 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba16dc8e-8ff5-4b5f-b285-e4032ffd76f2 admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:24:06.423588 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ef2478-586e-4c32-8398-fd0a0d44666c admin admin] PATH_INFO: `/v3/projects` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:06.424536 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ef2478-586e-4c32-8398-fd0a0d44666c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:24:06.426743 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ef2478-586e-4c32-8398-fd0a0d44666c admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:24:06.429870 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e8d6c9-e950-4c56-b2e4-f2e4a1cfc249 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:24:06.430680 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e8d6c9-e950-4c56-b2e4-f2e4a1cfc249 admin admin] REQUEST_METHOD: `PUT` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:06.431061 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e8d6c9-e950-4c56-b2e4-f2e4a1cfc249 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:06.431415 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e8d6c9-e950-4c56-b2e4-f2e4a1cfc249 admin admin] PATH_INFO: `/v3/projects/7a670696e63446e4bf597c40c6e72490/users/4ead4be9666942f2b81cbb0f9ab69325/roles/0679d6628a25438d90dc4bdc0401eda2` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:06.458177 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e8d6c9-e950-4c56-b2e4-f2e4a1cfc249 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7a670696e63446e4bf597c40c6e72490, user_id=4ead4be9666942f2b81cbb0f9ab69325, role_id=0679d6628a25438d90dc4bdc0401eda2)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:24:06.461527 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e8d6c9-e950-4c56-b2e4-f2e4a1cfc249 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:24:06.465953 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 807/1604] 10.222.0.15 () {60 vars in 1153 bytes} [Thu Sep 15 12:24:06 2022] POST /identity/v3/projects => generated 403 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 12:24:06.467113 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ef2478-586e-4c32-8398-fd0a0d44666c admin admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:24:06.490053 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 798/1605] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 15 12:24:06 2022] PUT /identity/v3/projects/7a670696e63446e4bf597c40c6e72490/users/4ead4be9666942f2b81cbb0f9ab69325/roles/0679d6628a25438d90dc4bdc0401eda2 => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:24:06.490053 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e8d6c9-e950-4c56-b2e4-f2e4a1cfc249 admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:24:06.549554 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ef2478-586e-4c32-8398-fd0a0d44666c admin admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:24:06.601452 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:24:06.609734 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e8d6c9-e950-4c56-b2e4-f2e4a1cfc249 admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:24:06.621303 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee63f389-813c-46cf-a700-7b92f9bf4a09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:24:06.622491 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee63f389-813c-46cf-a700-7b92f9bf4a09 admin admin] REQUEST_METHOD: `PUT` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:06.622919 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee63f389-813c-46cf-a700-7b92f9bf4a09 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:06.623401 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee63f389-813c-46cf-a700-7b92f9bf4a09 admin admin] PATH_INFO: `/v3/projects/d2780dab9a5d40739fc9adc9a6ce35a1/users/89cc4de76c6a4305af9a9506840e13f6/roles/0679d6628a25438d90dc4bdc0401eda2` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:06.641618 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee63f389-813c-46cf-a700-7b92f9bf4a09 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d2780dab9a5d40739fc9adc9a6ce35a1, user_id=89cc4de76c6a4305af9a9506840e13f6, role_id=0679d6628a25438d90dc4bdc0401eda2)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:24:06.646336 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee63f389-813c-46cf-a700-7b92f9bf4a09 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:24:06.652523 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3016f4be-1c98-44f4-bb2c-ec15b562c3bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:24:06.653916 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3016f4be-1c98-44f4-bb2c-ec15b562c3bc admin admin] REQUEST_METHOD: `PUT` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:06.653916 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3016f4be-1c98-44f4-bb2c-ec15b562c3bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:06.653916 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3016f4be-1c98-44f4-bb2c-ec15b562c3bc admin admin] PATH_INFO: `/v3/projects/59ace7b38143442bb6a511c14cb13a9a/users/7b8262b6c3a04914a89ed3548af3bf7b/roles/0679d6628a25438d90dc4bdc0401eda2` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:06.667003 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 799/1606] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 15 12:24:06 2022] PUT /identity/v3/projects/d2780dab9a5d40739fc9adc9a6ce35a1/users/89cc4de76c6a4305af9a9506840e13f6/roles/0679d6628a25438d90dc4bdc0401eda2 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:24:06.668107 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee63f389-813c-46cf-a700-7b92f9bf4a09 admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:24:06.680234 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3016f4be-1c98-44f4-bb2c-ec15b562c3bc admin admin] RBAC: Authorizing `identity:create_grant(project_id=59ace7b38143442bb6a511c14cb13a9a, user_id=7b8262b6c3a04914a89ed3548af3bf7b, role_id=0679d6628a25438d90dc4bdc0401eda2)` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:24:06.681276 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:24:06.687184 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3016f4be-1c98-44f4-bb2c-ec15b562c3bc admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:24:06.713052 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 808/1607] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 15 12:24:06 2022] PUT /identity/v3/projects/59ace7b38143442bb6a511c14cb13a9a/users/7b8262b6c3a04914a89ed3548af3bf7b/roles/0679d6628a25438d90dc4bdc0401eda2 => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:24:06.714510 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3016f4be-1c98-44f4-bb2c-ec15b562c3bc admin admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:24:06.781567 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee63f389-813c-46cf-a700-7b92f9bf4a09 admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:24:06.807753 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3016f4be-1c98-44f4-bb2c-ec15b562c3bc admin admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:24:06.813539 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1584e0-7f06-4e9b-a72b-ab57b42fd79d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:24:06.814422 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1584e0-7f06-4e9b-a72b-ab57b42fd79d admin admin] REQUEST_METHOD: `PUT` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:06.814806 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1584e0-7f06-4e9b-a72b-ab57b42fd79d admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:06.815180 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1584e0-7f06-4e9b-a72b-ab57b42fd79d admin admin] PATH_INFO: `/v3/projects/a0d7744eae56407d9fb9a126aa2b712c/users/d8bbf7fbe7e74d058e4f8541f12c10c1/roles/0679d6628a25438d90dc4bdc0401eda2` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:06.832644 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1584e0-7f06-4e9b-a72b-ab57b42fd79d admin admin] RBAC: Authorizing `identity:create_grant(project_id=a0d7744eae56407d9fb9a126aa2b712c, user_id=d8bbf7fbe7e74d058e4f8541f12c10c1, role_id=0679d6628a25438d90dc4bdc0401eda2)` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:24:06.837130 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1584e0-7f06-4e9b-a72b-ab57b42fd79d admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:24:06.857815 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 809/1608] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 15 12:24:06 2022] PUT /identity/v3/projects/a0d7744eae56407d9fb9a126aa2b712c/users/d8bbf7fbe7e74d058e4f8541f12c10c1/roles/0679d6628a25438d90dc4bdc0401eda2 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:24:06.858905 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1584e0-7f06-4e9b-a72b-ab57b42fd79d admin admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:24:06.884564 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ccc2194-943d-4d21-9821-e29877f69501 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:24:06.885363 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccc2194-943d-4d21-9821-e29877f69501 admin admin] REQUEST_METHOD: `PUT` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:06.885678 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccc2194-943d-4d21-9821-e29877f69501 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:06.885949 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccc2194-943d-4d21-9821-e29877f69501 admin admin] PATH_INFO: `/v3/projects/df56dce262544960bbba4f9308f98fc2/users/c5f9d95061464bee9b3a0cd7d695106d/roles/0679d6628a25438d90dc4bdc0401eda2` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:06.905595 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ccc2194-943d-4d21-9821-e29877f69501 admin admin] RBAC: Authorizing `identity:create_grant(project_id=df56dce262544960bbba4f9308f98fc2, user_id=c5f9d95061464bee9b3a0cd7d695106d, role_id=0679d6628a25438d90dc4bdc0401eda2)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:24:06.909012 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ccc2194-943d-4d21-9821-e29877f69501 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:24:06.945472 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 800/1609] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 15 12:24:06 2022] PUT /identity/v3/projects/df56dce262544960bbba4f9308f98fc2/users/c5f9d95061464bee9b3a0cd7d695106d/roles/0679d6628a25438d90dc4bdc0401eda2 => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:24:06.946950 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb0e53b-4545-4441-9f08-fb6baccd6e55 None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:06.947365 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb0e53b-4545-4441-9f08-fb6baccd6e55 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:06.947650 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb0e53b-4545-4441-9f08-fb6baccd6e55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:06.972896 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1584e0-7f06-4e9b-a72b-ab57b42fd79d admin admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:24:06.982675 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-fcb0e53b-4545-4441-9f08-fb6baccd6e55 None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:24:06.984929 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-fcb0e53b-4545-4441-9f08-fb6baccd6e55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:24:07.027476 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:24:07.049406 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc47a3a6-c5e6-42eb-8136-bf7a7c71caec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:24:07.050272 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc47a3a6-c5e6-42eb-8136-bf7a7c71caec admin admin] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:07.050617 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc47a3a6-c5e6-42eb-8136-bf7a7c71caec admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:07.050952 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc47a3a6-c5e6-42eb-8136-bf7a7c71caec admin admin] PATH_INFO: `/v3/users` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:07.051802 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc47a3a6-c5e6-42eb-8136-bf7a7c71caec admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:24:07.054015 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc47a3a6-c5e6-42eb-8136-bf7a7c71caec admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:24:07.085797 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 810/1610] 10.222.0.15 () {60 vars in 1144 bytes} [Thu Sep 15 12:24:06 2022] POST /identity/v3/users => generated 308 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 12:24:07.085797 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6074e5b-1273-4fa8-b928-b9ddc3e3f763 None None] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:07.085797 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6074e5b-1273-4fa8-b928-b9ddc3e3f763 None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:07.085797 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6074e5b-1273-4fa8-b928-b9ddc3e3f763 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:07.089196 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 801/1611] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 12:24:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:24:07.093682 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebbbdb8-af34-45a0-a28c-7c8fd3946d7d None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:07.093993 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebbbdb8-af34-45a0-a28c-7c8fd3946d7d None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:07.094254 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebbbdb8-af34-45a0-a28c-7c8fd3946d7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:07.131014 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.auth.core [None req-b6074e5b-1273-4fa8-b928-b9ddc3e3f763 None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81449) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:24:07.133332 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-b6074e5b-1273-4fa8-b928-b9ddc3e3f763 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:24:07.149706 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 811/1612] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 12:24:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 66 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:24:07.151045 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627b62e2-0bce-4da2-aabb-14cd860220ae None None] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:07.151864 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627b62e2-0bce-4da2-aabb-14cd860220ae None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:07.152133 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627b62e2-0bce-4da2-aabb-14cd860220ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:07.156931 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-6ebbbdb8-af34-45a0-a28c-7c8fd3946d7d None None] MFA Rules not processed for user `7b8262b6c3a04914a89ed3548af3bf7b`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:24:07.158166 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-6ebbbdb8-af34-45a0-a28c-7c8fd3946d7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:24:07.203558 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.auth.core [None req-627b62e2-0bce-4da2-aabb-14cd860220ae None None] MFA Rules not processed for user `d8bbf7fbe7e74d058e4f8541f12c10c1`. Rule list: `[]` (Enabled: `True`). {{(pid=81449) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:24:07.206409 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-627b62e2-0bce-4da2-aabb-14cd860220ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:24:07.233348 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:24:07.306904 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 802/1613] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 12:24:07 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:24:07.314402 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49306dd7-f10a-4488-870c-33d8b3b9916b None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:07.314692 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49306dd7-f10a-4488-870c-33d8b3b9916b None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:07.314972 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49306dd7-f10a-4488-870c-33d8b3b9916b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:07.358496 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-49306dd7-f10a-4488-870c-33d8b3b9916b None None] MFA Rules not processed for user `c5f9d95061464bee9b3a0cd7d695106d`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:24:07.359964 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-49306dd7-f10a-4488-870c-33d8b3b9916b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:24:07.391332 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 812/1614] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 12:24:07 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:24:07.392847 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627b62e2-0bce-4da2-aabb-14cd860220ae None None] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:24:07.428430 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627b62e2-0bce-4da2-aabb-14cd860220ae None None] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:24:07.449577 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0bba92b-0289-43b4-bb2e-d15d3b56c2fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:24:07.450484 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bba92b-0289-43b4-bb2e-d15d3b56c2fa admin admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:07.450950 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bba92b-0289-43b4-bb2e-d15d3b56c2fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:07.451367 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bba92b-0289-43b4-bb2e-d15d3b56c2fa admin admin] PATH_INFO: `/v3/roles` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:07.452173 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0bba92b-0289-43b4-bb2e-d15d3b56c2fa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:24:07.453994 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0bba92b-0289-43b4-bb2e-d15d3b56c2fa admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:24:07.468425 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 813/1615] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 12:24:07 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 12:24:07.478302 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0bba92b-0289-43b4-bb2e-d15d3b56c2fa admin admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:24:07.512869 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 803/1616] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 12:24:07 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:24:07.513898 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49306dd7-f10a-4488-870c-33d8b3b9916b None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:24:07.608615 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49306dd7-f10a-4488-870c-33d8b3b9916b None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:24:07.615905 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec664463-d6e5-47c0-84e5-800d789f65c4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfbb4d6809064f9cb9e06626ed7d6814', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '540a32ef88764e8c8d9201ae22f6ddaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:24:07.616575 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec664463-d6e5-47c0-84e5-800d789f65c4 service neutron] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:07.617014 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec664463-d6e5-47c0-84e5-800d789f65c4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:07.617341 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec664463-d6e5-47c0-84e5-800d789f65c4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:07.619444 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0bba92b-0289-43b4-bb2e-d15d3b56c2fa admin admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:24:07.625279 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ace455-a29b-4b9e-9527-83b9614b8e4c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfbb4d6809064f9cb9e06626ed7d6814', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '540a32ef88764e8c8d9201ae22f6ddaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:24:07.625916 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ace455-a29b-4b9e-9527-83b9614b8e4c service neutron] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:07.626197 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ace455-a29b-4b9e-9527-83b9614b8e4c service neutron] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:07.626469 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ace455-a29b-4b9e-9527-83b9614b8e4c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:07.629029 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec664463-d6e5-47c0-84e5-800d789f65c4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:24:07.630992 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec664463-d6e5-47c0-84e5-800d789f65c4 service neutron] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:24:07.636787 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ace455-a29b-4b9e-9527-83b9614b8e4c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:24:07.638799 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ace455-a29b-4b9e-9527-83b9614b8e4c service neutron] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:24:07.644738 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 804/1617] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 12:24:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 12:24:07.646982 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba87605-1c5b-47c4-82fd-456a4548e0fe None None] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:07.647597 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba87605-1c5b-47c4-82fd-456a4548e0fe None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:07.648554 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba87605-1c5b-47c4-82fd-456a4548e0fe None None] PATH_INFO: `/` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:07.650116 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 805/1618] 10.222.0.15 () {54 vars in 962 bytes} [Thu Sep 15 12:24:07 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:24:07.650341 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 814/1619] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 12:24:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 12:24:07.651769 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ace455-a29b-4b9e-9527-83b9614b8e4c service neutron] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:24:07.658048 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba87605-1c5b-47c4-82fd-456a4548e0fe None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:24:07.672321 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba87605-1c5b-47c4-82fd-456a4548e0fe None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:24:07.678007 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98164fb2-1a7c-4b3a-b21d-25e5c2795f36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:24:07.679049 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98164fb2-1a7c-4b3a-b21d-25e5c2795f36 admin admin] REQUEST_METHOD: `PUT` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:07.679481 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98164fb2-1a7c-4b3a-b21d-25e5c2795f36 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:07.680081 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98164fb2-1a7c-4b3a-b21d-25e5c2795f36 admin admin] PATH_INFO: `/v3/projects/eece3bf66bcb4765b9e94a2f1a0ed1e5/users/934491e7f02a4f14be2aca5de0b05a8b/roles/9c769b5c82884766b2c510b1f0f1364c` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:07.720543 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98164fb2-1a7c-4b3a-b21d-25e5c2795f36 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eece3bf66bcb4765b9e94a2f1a0ed1e5, user_id=934491e7f02a4f14be2aca5de0b05a8b, role_id=9c769b5c82884766b2c510b1f0f1364c)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:24:07.729295 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98164fb2-1a7c-4b3a-b21d-25e5c2795f36 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:24:07.735366 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:24:07.743144 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ace455-a29b-4b9e-9527-83b9614b8e4c service neutron] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:24:07.754341 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121fb075-7c1f-4996-ac76-cbe65bb80851 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8191d3ccb874529b32f8d0026db629d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '540a32ef88764e8c8d9201ae22f6ddaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:24:07.755244 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121fb075-7c1f-4996-ac76-cbe65bb80851 service cinder] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:07.755741 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121fb075-7c1f-4996-ac76-cbe65bb80851 service cinder] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:07.756592 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121fb075-7c1f-4996-ac76-cbe65bb80851 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:07.762630 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 806/1620] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 15 12:24:07 2022] PUT /identity/v3/projects/eece3bf66bcb4765b9e94a2f1a0ed1e5/users/934491e7f02a4f14be2aca5de0b05a8b/roles/9c769b5c82884766b2c510b1f0f1364c => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:24:07.763489 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98164fb2-1a7c-4b3a-b21d-25e5c2795f36 admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:24:07.781090 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121fb075-7c1f-4996-ac76-cbe65bb80851 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:24:07.787021 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121fb075-7c1f-4996-ac76-cbe65bb80851 service cinder] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:24:07.814899 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 815/1621] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 12:24:07 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 12:24:07.819307 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089576cf-f652-43c0-aded-b6112f7a6263 None None] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:07.819697 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089576cf-f652-43c0-aded-b6112f7a6263 None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:07.820181 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089576cf-f652-43c0-aded-b6112f7a6263 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:07.865230 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:24:07.912307 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.auth.core [None req-089576cf-f652-43c0-aded-b6112f7a6263 None None] MFA Rules not processed for user `eface89e261840a2abb4206be26455c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81449) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:24:07.917744 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-089576cf-f652-43c0-aded-b6112f7a6263 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:24:07.931100 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98164fb2-1a7c-4b3a-b21d-25e5c2795f36 admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:24:07.936861 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12cc518-60da-48c7-af50-a9f09b263f3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:24:07.937484 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12cc518-60da-48c7-af50-a9f09b263f3d admin admin] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:07.937816 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12cc518-60da-48c7-af50-a9f09b263f3d admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:07.938156 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a12cc518-60da-48c7-af50-a9f09b263f3d admin admin] PATH_INFO: `/v3/projects` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:07.938897 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12cc518-60da-48c7-af50-a9f09b263f3d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:24:07.941168 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a12cc518-60da-48c7-af50-a9f09b263f3d admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:24:07.988207 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 807/1622] 10.222.0.15 () {60 vars in 1153 bytes} [Thu Sep 15 12:24:07 2022] POST /identity/v3/projects => generated 373 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 12:24:07.991644 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12cc518-60da-48c7-af50-a9f09b263f3d admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:24:08.132505 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 816/1623] 10.222.0.15 () {58 vars in 1059 bytes} [Thu Sep 15 12:24:07 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 317 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:24:08.144013 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089576cf-f652-43c0-aded-b6112f7a6263 None None] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:24:08.158057 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089576cf-f652-43c0-aded-b6112f7a6263 None None] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:24:08.166671 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a12cc518-60da-48c7-af50-a9f09b263f3d admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:24:08.176161 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f022f8-80ef-41db-9010-a656403a579e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:24:08.176844 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f022f8-80ef-41db-9010-a656403a579e admin admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:08.177198 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f022f8-80ef-41db-9010-a656403a579e admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:08.177483 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f022f8-80ef-41db-9010-a656403a579e admin admin] PATH_INFO: `/v3/roles` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:08.178120 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f022f8-80ef-41db-9010-a656403a579e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:24:08.179261 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15a927a-0cdc-4020-8681-1ad862e40916 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:24:08.179728 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f022f8-80ef-41db-9010-a656403a579e admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:24:08.180429 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15a927a-0cdc-4020-8681-1ad862e40916 admin admin] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:08.181718 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15a927a-0cdc-4020-8681-1ad862e40916 admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:08.183037 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15a927a-0cdc-4020-8681-1ad862e40916 admin admin] PATH_INFO: `/v3/users` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:08.185381 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15a927a-0cdc-4020-8681-1ad862e40916 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:24:08.188777 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15a927a-0cdc-4020-8681-1ad862e40916 admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:24:08.203768 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 808/1624] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 12:24:07 2022] GET /identity/v3/roles => generated 1420 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 12:24:08.204560 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f022f8-80ef-41db-9010-a656403a579e admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:24:08.233941 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 817/1625] 10.222.0.15 () {60 vars in 1144 bytes} [Thu Sep 15 12:24:08 2022] POST /identity/v3/users => generated 293 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 12:24:08.235084 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15a927a-0cdc-4020-8681-1ad862e40916 admin admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:24:08.254271 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15a927a-0cdc-4020-8681-1ad862e40916 admin admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:24:08.263090 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7f820f-9b22-4f45-a08c-3e492d1962af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:24:08.264288 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7f820f-9b22-4f45-a08c-3e492d1962af admin admin] REQUEST_METHOD: `PUT` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:08.264288 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7f820f-9b22-4f45-a08c-3e492d1962af admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:08.264564 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7f820f-9b22-4f45-a08c-3e492d1962af admin admin] PATH_INFO: `/v3/projects/eece3bf66bcb4765b9e94a2f1a0ed1e5/users/934491e7f02a4f14be2aca5de0b05a8b/roles/9c769b5c82884766b2c510b1f0f1364c` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:08.266671 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7f820f-9b22-4f45-a08c-3e492d1962af admin admin] RBAC: Authorizing `identity:create_grant(project_id=eece3bf66bcb4765b9e94a2f1a0ed1e5, user_id=934491e7f02a4f14be2aca5de0b05a8b, role_id=9c769b5c82884766b2c510b1f0f1364c)` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:24:08.270132 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7f820f-9b22-4f45-a08c-3e492d1962af admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:24:08.287315 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 818/1626] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 15 12:24:08 2022] PUT /identity/v3/projects/eece3bf66bcb4765b9e94a2f1a0ed1e5/users/934491e7f02a4f14be2aca5de0b05a8b/roles/9c769b5c82884766b2c510b1f0f1364c => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:24:08.288288 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7f820f-9b22-4f45-a08c-3e492d1962af admin admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:24:08.368657 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f022f8-80ef-41db-9010-a656403a579e admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:24:08.373295 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7f820f-9b22-4f45-a08c-3e492d1962af admin admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:24:08.383434 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68dd509e-9c75-4131-95b8-3331d917fe07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:24:08.384063 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68dd509e-9c75-4131-95b8-3331d917fe07 admin admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:08.384343 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68dd509e-9c75-4131-95b8-3331d917fe07 admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:08.386518 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68dd509e-9c75-4131-95b8-3331d917fe07 admin admin] PATH_INFO: `/v3/roles` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:08.390001 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68dd509e-9c75-4131-95b8-3331d917fe07 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:24:08.392854 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68dd509e-9c75-4131-95b8-3331d917fe07 admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:24:08.420789 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 819/1627] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 12:24:08 2022] GET /identity/v3/roles => generated 1420 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 12:24:08.424059 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13d7202-6b78-4153-9c7c-d45438804f9d None None] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:08.424352 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13d7202-6b78-4153-9c7c-d45438804f9d None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:08.425792 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13d7202-6b78-4153-9c7c-d45438804f9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:08.499579 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:24:08.502226 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.auth.core [None req-a13d7202-6b78-4153-9c7c-d45438804f9d None None] MFA Rules not processed for user `934491e7f02a4f14be2aca5de0b05a8b`. Rule list: `[]` (Enabled: `True`). {{(pid=81449) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:24:08.506593 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-a13d7202-6b78-4153-9c7c-d45438804f9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:24:08.537805 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28831ec0-8493-4b42-8317-5277af0fdc4e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eface89e261840a2abb4206be26455c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '540a32ef88764e8c8d9201ae22f6ddaf', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:24:08.540785 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28831ec0-8493-4b42-8317-5277af0fdc4e service nova] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:08.541305 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28831ec0-8493-4b42-8317-5277af0fdc4e service nova] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:08.541583 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28831ec0-8493-4b42-8317-5277af0fdc4e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:08.570187 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:24:08.712011 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28831ec0-8493-4b42-8317-5277af0fdc4e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:24:08.714865 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28831ec0-8493-4b42-8317-5277af0fdc4e service nova] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:24:08.718958 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 820/1628] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 12:24:08 2022] POST /identity/v3/auth/tokens => generated 3347 bytes in 297 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:24:08.719865 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13d7202-6b78-4153-9c7c-d45438804f9d None None] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:24:08.740728 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 809/1629] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 12:24:08 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 537 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 12:24:08.741653 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28831ec0-8493-4b42-8317-5277af0fdc4e service nova] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:24:08.741778 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13d7202-6b78-4153-9c7c-d45438804f9d None None] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:24:08.754250 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09bf8c29-f3a4-43c6-b37f-c6175cc109f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:24:08.754955 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bf8c29-f3a4-43c6-b37f-c6175cc109f4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:08.755236 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bf8c29-f3a4-43c6-b37f-c6175cc109f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:08.755509 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bf8c29-f3a4-43c6-b37f-c6175cc109f4 admin admin] PATH_INFO: `/v3/projects/b16dfc33fb784887b773d73227fe99b2/users/abd7a183d2434dc8b864634cd84c9d4e/roles/9c769b5c82884766b2c510b1f0f1364c` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:08.789101 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09bf8c29-f3a4-43c6-b37f-c6175cc109f4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b16dfc33fb784887b773d73227fe99b2, user_id=abd7a183d2434dc8b864634cd84c9d4e, role_id=9c769b5c82884766b2c510b1f0f1364c)` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:24:08.793273 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09bf8c29-f3a4-43c6-b37f-c6175cc109f4 admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:24:08.820225 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 821/1630] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 15 12:24:08 2022] PUT /identity/v3/projects/b16dfc33fb784887b773d73227fe99b2/users/abd7a183d2434dc8b864634cd84c9d4e/roles/9c769b5c82884766b2c510b1f0f1364c => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:24:08.825009 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09bf8c29-f3a4-43c6-b37f-c6175cc109f4 admin admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:24:08.873089 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28831ec0-8493-4b42-8317-5277af0fdc4e service nova] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:24:09.002027 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09bf8c29-f3a4-43c6-b37f-c6175cc109f4 admin admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:24:09.040395 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31aee6af-f7a7-4706-ae7d-59fe08d967f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:24:09.043620 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31aee6af-f7a7-4706-ae7d-59fe08d967f4 admin admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:09.045118 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31aee6af-f7a7-4706-ae7d-59fe08d967f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:09.045645 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31aee6af-f7a7-4706-ae7d-59fe08d967f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:09.054209 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31aee6af-f7a7-4706-ae7d-59fe08d967f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:24:09.056165 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31aee6af-f7a7-4706-ae7d-59fe08d967f4 admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:24:09.079138 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 822/1631] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 12:24:08 2022] GET /identity/v3/roles => generated 1420 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 12:24:09.083206 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31aee6af-f7a7-4706-ae7d-59fe08d967f4 admin admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:24:09.100077 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31aee6af-f7a7-4706-ae7d-59fe08d967f4 admin admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:24:09.103441 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f765fac-2bf0-45e8-8f34-336c160d8f75 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8191d3ccb874529b32f8d0026db629d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '540a32ef88764e8c8d9201ae22f6ddaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:24:09.104768 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f765fac-2bf0-45e8-8f34-336c160d8f75 service cinder] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:09.105148 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f765fac-2bf0-45e8-8f34-336c160d8f75 service cinder] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:09.105847 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f765fac-2bf0-45e8-8f34-336c160d8f75 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:09.114675 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6f9980-4218-4435-b1f4-7dcb4dafab92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:24:09.115902 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6f9980-4218-4435-b1f4-7dcb4dafab92 admin admin] REQUEST_METHOD: `PUT` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:09.116392 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6f9980-4218-4435-b1f4-7dcb4dafab92 admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:09.116892 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6f9980-4218-4435-b1f4-7dcb4dafab92 admin admin] PATH_INFO: `/v3/projects/b16dfc33fb784887b773d73227fe99b2/users/abd7a183d2434dc8b864634cd84c9d4e/roles/9c769b5c82884766b2c510b1f0f1364c` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:09.122233 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6f9980-4218-4435-b1f4-7dcb4dafab92 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b16dfc33fb784887b773d73227fe99b2, user_id=abd7a183d2434dc8b864634cd84c9d4e, role_id=9c769b5c82884766b2c510b1f0f1364c)` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:24:09.126518 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6f9980-4218-4435-b1f4-7dcb4dafab92 admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:24:09.160158 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 823/1632] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 15 12:24:09 2022] PUT /identity/v3/projects/b16dfc33fb784887b773d73227fe99b2/users/abd7a183d2434dc8b864634cd84c9d4e/roles/9c769b5c82884766b2c510b1f0f1364c => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:24:09.170328 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d166514d-70d3-4652-863e-8efad4e929ce None None] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:09.170878 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d166514d-70d3-4652-863e-8efad4e929ce None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:09.171376 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d166514d-70d3-4652-863e-8efad4e929ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:09.269745 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.auth.core [None req-d166514d-70d3-4652-863e-8efad4e929ce None None] MFA Rules not processed for user `abd7a183d2434dc8b864634cd84c9d4e`. Rule list: `[]` (Enabled: `True`). {{(pid=81449) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:24:09.272884 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-d166514d-70d3-4652-863e-8efad4e929ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:24:09.282718 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f765fac-2bf0-45e8-8f34-336c160d8f75 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:24:09.286124 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f765fac-2bf0-45e8-8f34-336c160d8f75 service cinder] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:24:09.296004 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:24:09.324644 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 810/1633] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 12:24:08 2022] GET /identity/v3/auth/tokens => generated 3347 bytes in 582 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 12:24:09.326803 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b6f485-b510-4d7f-9c4e-9b00d6b9aabf None None] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:09.327176 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b6f485-b510-4d7f-9c4e-9b00d6b9aabf None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:09.327515 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b6f485-b510-4d7f-9c4e-9b00d6b9aabf None None] PATH_INFO: `/` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:09.338262 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 811/1634] 10.222.0.15 () {54 vars in 912 bytes} [Thu Sep 15 12:24:09 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:24:09.344707 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f7ab65-6d7b-4b99-a517-bf3c13b6b13d None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:09.345273 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f7ab65-6d7b-4b99-a517-bf3c13b6b13d None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:09.346038 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f7ab65-6d7b-4b99-a517-bf3c13b6b13d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:09.385260 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-65f7ab65-6d7b-4b99-a517-bf3c13b6b13d None None] MFA Rules not processed for user `eface89e261840a2abb4206be26455c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:24:09.388522 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-65f7ab65-6d7b-4b99-a517-bf3c13b6b13d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:24:09.514591 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 824/1635] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 12:24:09 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 345 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:24:09.534141 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d166514d-70d3-4652-863e-8efad4e929ce None None] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:24:09.534141 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:24:09.566090 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 812/1636] 10.222.0.15 () {58 vars in 1009 bytes} [Thu Sep 15 12:24:09 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:24:09.655021 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f7ab65-6d7b-4b99-a517-bf3c13b6b13d None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:24:09.711907 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d166514d-70d3-4652-863e-8efad4e929ce None None] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:24:09.721679 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf23d71-cc29-45ca-bb29-cba1460a12d7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8191d3ccb874529b32f8d0026db629d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '540a32ef88764e8c8d9201ae22f6ddaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:24:09.722805 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf23d71-cc29-45ca-bb29-cba1460a12d7 service cinder] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:09.723149 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf23d71-cc29-45ca-bb29-cba1460a12d7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:09.723486 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf23d71-cc29-45ca-bb29-cba1460a12d7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:09.739039 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf23d71-cc29-45ca-bb29-cba1460a12d7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:24:09.740670 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf23d71-cc29-45ca-bb29-cba1460a12d7 service cinder] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:24:09.756602 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 825/1637] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 12:24:09 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 229 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 12:24:09.822605 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f7ab65-6d7b-4b99-a517-bf3c13b6b13d None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:24:09.830843 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2892dfc-ebf8-42ab-b292-27eece27649e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8191d3ccb874529b32f8d0026db629d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '540a32ef88764e8c8d9201ae22f6ddaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:24:09.832013 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2892dfc-ebf8-42ab-b292-27eece27649e service cinder] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:09.832300 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2892dfc-ebf8-42ab-b292-27eece27649e service cinder] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:09.832566 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2892dfc-ebf8-42ab-b292-27eece27649e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:09.849057 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2892dfc-ebf8-42ab-b292-27eece27649e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:24:09.850566 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2892dfc-ebf8-42ab-b292-27eece27649e service cinder] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:24:09.881658 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 813/1638] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 12:24:09 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 220 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 12:24:09.970214 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf23d71-cc29-45ca-bb29-cba1460a12d7 service cinder] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:24:10.178228 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf23d71-cc29-45ca-bb29-cba1460a12d7 service cinder] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:24:10.192097 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc992c0-71d0-4f16-a753-5aedd4567357 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:24:10.193987 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc992c0-71d0-4f16-a753-5aedd4567357 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:10.194339 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc992c0-71d0-4f16-a753-5aedd4567357 admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:10.195771 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc992c0-71d0-4f16-a753-5aedd4567357 admin admin] PATH_INFO: `/v3/users/d4ea50ffefa34b17808f3458fb0654d6` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:10.198857 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc992c0-71d0-4f16-a753-5aedd4567357 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d4ea50ffefa34b17808f3458fb0654d6)` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:24:10.203582 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc992c0-71d0-4f16-a753-5aedd4567357 admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:24:10.359935 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-1fc992c0-71d0-4f16-a753-5aedd4567357 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd4ea50ffefa34b17808f3458fb0654d6'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:24:10.384732 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-1fc992c0-71d0-4f16-a753-5aedd4567357 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd4ea50ffefa34b17808f3458fb0654d6'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:24:10.414435 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-1fc992c0-71d0-4f16-a753-5aedd4567357 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd4ea50ffefa34b17808f3458fb0654d6'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:24:10.419957 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-1fc992c0-71d0-4f16-a753-5aedd4567357 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd4ea50ffefa34b17808f3458fb0654d6'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:24:10.444789 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-1fc992c0-71d0-4f16-a753-5aedd4567357 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd4ea50ffefa34b17808f3458fb0654d6'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:24:10.464273 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 826/1639] 10.222.0.15 () {58 vars in 1224 bytes} [Thu Sep 15 12:24:09 2022] DELETE /identity/v3/users/d4ea50ffefa34b17808f3458fb0654d6 => generated 0 bytes in 492 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:24:10.466058 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e92d54-264e-4009-9aaf-1d1374671ee3 None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:10.466554 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e92d54-264e-4009-9aaf-1d1374671ee3 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:10.466667 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e92d54-264e-4009-9aaf-1d1374671ee3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:10.513896 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-77e92d54-264e-4009-9aaf-1d1374671ee3 None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:24:10.515359 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-77e92d54-264e-4009-9aaf-1d1374671ee3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:24:10.732899 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 814/1640] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 12:24:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 268 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:24:10.756090 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc992c0-71d0-4f16-a753-5aedd4567357 admin admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:24:10.757561 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-1fc992c0-71d0-4f16-a753-5aedd4567357 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:24:10.835609 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:24:10.930911 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc992c0-71d0-4f16-a753-5aedd4567357 admin admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:24:10.946312 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540c511d-9468-4957-a9ff-7f735f6831a6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfbb4d6809064f9cb9e06626ed7d6814', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '540a32ef88764e8c8d9201ae22f6ddaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:24:10.946509 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540c511d-9468-4957-a9ff-7f735f6831a6 service neutron] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:10.947266 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540c511d-9468-4957-a9ff-7f735f6831a6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:10.947266 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540c511d-9468-4957-a9ff-7f735f6831a6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:10.966854 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540c511d-9468-4957-a9ff-7f735f6831a6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:24:10.972264 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540c511d-9468-4957-a9ff-7f735f6831a6 service neutron] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:24:11.009543 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 827/1641] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 12:24:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 254 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 12:24:11.092129 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e92d54-264e-4009-9aaf-1d1374671ee3 None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:24:11.093608 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-77e92d54-264e-4009-9aaf-1d1374671ee3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:24:11.117948 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e92d54-264e-4009-9aaf-1d1374671ee3 None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:24:11.125823 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18aa3d61-bfad-49b9-9b52-be5e106d61dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:24:11.126430 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18aa3d61-bfad-49b9-9b52-be5e106d61dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:11.126707 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18aa3d61-bfad-49b9-9b52-be5e106d61dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:11.127019 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18aa3d61-bfad-49b9-9b52-be5e106d61dc admin admin] PATH_INFO: `/v3/users/c5f9d95061464bee9b3a0cd7d695106d` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:11.129866 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18aa3d61-bfad-49b9-9b52-be5e106d61dc admin admin] RBAC: Authorizing `identity:delete_user(user_id=c5f9d95061464bee9b3a0cd7d695106d)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:24:11.130906 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: WARNING py.warnings [None req-18aa3d61-bfad-49b9-9b52-be5e106d61dc admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 15 12:24:11.130906 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: warnings.warn(msg) Sep 15 12:24:11.130906 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: Sep 15 12:24:11.132469 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18aa3d61-bfad-49b9-9b52-be5e106d61dc admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:24:11.157732 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f201e0-00ee-445b-837f-2d303dee61f0 None None] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:11.158113 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f201e0-00ee-445b-837f-2d303dee61f0 None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:11.158460 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f201e0-00ee-445b-837f-2d303dee61f0 None None] PATH_INFO: `/` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:11.159929 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 828/1642] 10.222.0.15 () {54 vars in 962 bytes} [Thu Sep 15 12:24:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 15 12:24:11.173318 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84005539-8fe8-4680-82fa-9c6e752f1cf5 None None] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:11.173609 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84005539-8fe8-4680-82fa-9c6e752f1cf5 None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:11.174020 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84005539-8fe8-4680-82fa-9c6e752f1cf5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:11.227037 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.auth.core [None req-84005539-8fe8-4680-82fa-9c6e752f1cf5 None None] MFA Rules not processed for user `eface89e261840a2abb4206be26455c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81449) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:24:11.228360 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-84005539-8fe8-4680-82fa-9c6e752f1cf5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:24:11.286580 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:24:11.324866 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-18aa3d61-bfad-49b9-9b52-be5e106d61dc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c5f9d95061464bee9b3a0cd7d695106d'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:24:11.366346 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-18aa3d61-bfad-49b9-9b52-be5e106d61dc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c5f9d95061464bee9b3a0cd7d695106d'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:24:11.384993 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-18aa3d61-bfad-49b9-9b52-be5e106d61dc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c5f9d95061464bee9b3a0cd7d695106d'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:24:11.385752 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-18aa3d61-bfad-49b9-9b52-be5e106d61dc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c5f9d95061464bee9b3a0cd7d695106d'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:24:11.423953 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-18aa3d61-bfad-49b9-9b52-be5e106d61dc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c5f9d95061464bee9b3a0cd7d695106d'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:24:11.430942 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 829/1643] 10.222.0.15 () {58 vars in 1059 bytes} [Thu Sep 15 12:24:11 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 258 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 12:24:11.436159 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640b3c1f-09ef-429a-aa95-df27f7e678fe None None] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:11.436447 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640b3c1f-09ef-429a-aa95-df27f7e678fe None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:11.436714 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640b3c1f-09ef-429a-aa95-df27f7e678fe None None] PATH_INFO: `/` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:11.438215 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 830/1644] 10.222.0.15 () {54 vars in 912 bytes} [Thu Sep 15 12:24:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 15 12:24:11.438968 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 815/1645] 10.222.0.15 () {58 vars in 1224 bytes} [Thu Sep 15 12:24:11 2022] DELETE /identity/v3/users/c5f9d95061464bee9b3a0cd7d695106d => generated 0 bytes in 347 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:24:11.444183 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6739501f-34c1-4fa4-82fe-b15504519b5e None None] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:11.444601 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6739501f-34c1-4fa4-82fe-b15504519b5e None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:11.445013 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6739501f-34c1-4fa4-82fe-b15504519b5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:11.445171 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25b9245-a295-4b4d-9420-1e439a9eac17 None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:11.445470 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25b9245-a295-4b4d-9420-1e439a9eac17 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:11.446580 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25b9245-a295-4b4d-9420-1e439a9eac17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:11.470228 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-c25b9245-a295-4b4d-9420-1e439a9eac17 None None] MFA Rules not processed for user `eface89e261840a2abb4206be26455c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:24:11.472711 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-c25b9245-a295-4b4d-9420-1e439a9eac17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:24:11.493123 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.auth.core [None req-6739501f-34c1-4fa4-82fe-b15504519b5e None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81449) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:24:11.496894 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-6739501f-34c1-4fa4-82fe-b15504519b5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:24:11.639140 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 816/1646] 10.222.0.15 () {58 vars in 1009 bytes} [Thu Sep 15 12:24:11 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:24:11.645924 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 831/1647] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 12:24:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:24:11.660588 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25b9245-a295-4b4d-9420-1e439a9eac17 None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:24:11.663287 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-c25b9245-a295-4b4d-9420-1e439a9eac17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:24:11.675452 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6739501f-34c1-4fa4-82fe-b15504519b5e None None] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:24:11.677235 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-6739501f-34c1-4fa4-82fe-b15504519b5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:24:11.800919 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6739501f-34c1-4fa4-82fe-b15504519b5e None None] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:24:11.806807 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94249050-503a-44ca-acda-6507c1f8033e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfbb4d6809064f9cb9e06626ed7d6814', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '540a32ef88764e8c8d9201ae22f6ddaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:24:11.807611 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94249050-503a-44ca-acda-6507c1f8033e service neutron] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:11.808100 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94249050-503a-44ca-acda-6507c1f8033e service neutron] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:11.808616 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94249050-503a-44ca-acda-6507c1f8033e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:11.835987 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94249050-503a-44ca-acda-6507c1f8033e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:24:11.838226 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25b9245-a295-4b4d-9420-1e439a9eac17 None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:24:11.841878 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94249050-503a-44ca-acda-6507c1f8033e service neutron] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:24:11.843560 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539eab3a-3f35-4f34-81fb-bf32e5c3142c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8191d3ccb874529b32f8d0026db629d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '540a32ef88764e8c8d9201ae22f6ddaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:24:11.844478 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539eab3a-3f35-4f34-81fb-bf32e5c3142c service cinder] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:11.844937 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539eab3a-3f35-4f34-81fb-bf32e5c3142c service cinder] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:11.845342 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539eab3a-3f35-4f34-81fb-bf32e5c3142c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:11.860637 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-539eab3a-3f35-4f34-81fb-bf32e5c3142c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:24:11.862266 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-539eab3a-3f35-4f34-81fb-bf32e5c3142c service cinder] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:24:11.875211 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 832/1648] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 12:24:11 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 200 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 15 12:24:11.875762 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 817/1649] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 12:24:11 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 216 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 12:24:12.429048 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94249050-503a-44ca-acda-6507c1f8033e service neutron] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:24:12.429048 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-94249050-503a-44ca-acda-6507c1f8033e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:24:12.451938 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:24:12.468777 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94249050-503a-44ca-acda-6507c1f8033e service neutron] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:24:12.486653 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc83a1c-1eb4-4051-9cda-cf8df62dad6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:24:12.490413 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc83a1c-1eb4-4051-9cda-cf8df62dad6e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:12.491273 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc83a1c-1eb4-4051-9cda-cf8df62dad6e admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:12.492600 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc83a1c-1eb4-4051-9cda-cf8df62dad6e admin admin] PATH_INFO: `/v3/projects/3191251608ff40dd94189a51a45adf76` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:12.495898 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc83a1c-1eb4-4051-9cda-cf8df62dad6e admin admin] RBAC: Authorizing `identity:delete_project(project_id=3191251608ff40dd94189a51a45adf76)` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:24:12.501455 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: WARNING py.warnings [None req-fbc83a1c-1eb4-4051-9cda-cf8df62dad6e admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Sep 15 12:24:12.501455 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: warnings.warn(msg) Sep 15 12:24:12.501455 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: Sep 15 12:24:12.508186 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc83a1c-1eb4-4051-9cda-cf8df62dad6e admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:24:12.708684 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-fbc83a1c-1eb4-4051-9cda-cf8df62dad6e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3191251608ff40dd94189a51a45adf76', 'request_id': 'req-fbc83a1c-1eb4-4051-9cda-cf8df62dad6e'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:24:12.730201 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-fbc83a1c-1eb4-4051-9cda-cf8df62dad6e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3191251608ff40dd94189a51a45adf76', 'request_id': 'req-fbc83a1c-1eb4-4051-9cda-cf8df62dad6e'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:24:12.763255 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-fbc83a1c-1eb4-4051-9cda-cf8df62dad6e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3191251608ff40dd94189a51a45adf76', 'request_id': 'req-fbc83a1c-1eb4-4051-9cda-cf8df62dad6e'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:24:12.805688 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-fbc83a1c-1eb4-4051-9cda-cf8df62dad6e admin admin] The token cache is being invalidate because project 3191251608ff40dd94189a51a45adf76 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81449) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 12:24:12.806242 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-fbc83a1c-1eb4-4051-9cda-cf8df62dad6e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:24:12.806934 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-fbc83a1c-1eb4-4051-9cda-cf8df62dad6e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:24:12.812873 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 833/1650] 10.222.0.15 () {58 vars in 1233 bytes} [Thu Sep 15 12:24:12 2022] DELETE /identity/v3/projects/3191251608ff40dd94189a51a45adf76 => generated 0 bytes in 382 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:24:12.813007 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539eab3a-3f35-4f34-81fb-bf32e5c3142c service cinder] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:24:12.833511 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539eab3a-3f35-4f34-81fb-bf32e5c3142c service cinder] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:24:12.907910 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:24:12.950936 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650efaac-85ea-4d3b-8036-00f87ca4d8b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:24:12.953858 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650efaac-85ea-4d3b-8036-00f87ca4d8b5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:12.954266 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650efaac-85ea-4d3b-8036-00f87ca4d8b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:12.954557 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650efaac-85ea-4d3b-8036-00f87ca4d8b5 admin admin] PATH_INFO: `/v3/users/934491e7f02a4f14be2aca5de0b05a8b` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:12.957675 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650efaac-85ea-4d3b-8036-00f87ca4d8b5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=934491e7f02a4f14be2aca5de0b05a8b)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:24:12.965503 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650efaac-85ea-4d3b-8036-00f87ca4d8b5 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:24:13.066953 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-650efaac-85ea-4d3b-8036-00f87ca4d8b5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '934491e7f02a4f14be2aca5de0b05a8b'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:24:13.087725 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-650efaac-85ea-4d3b-8036-00f87ca4d8b5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '934491e7f02a4f14be2aca5de0b05a8b'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:24:13.098423 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-650efaac-85ea-4d3b-8036-00f87ca4d8b5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '934491e7f02a4f14be2aca5de0b05a8b'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:24:13.099535 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-650efaac-85ea-4d3b-8036-00f87ca4d8b5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '934491e7f02a4f14be2aca5de0b05a8b'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:24:13.130929 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-650efaac-85ea-4d3b-8036-00f87ca4d8b5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '934491e7f02a4f14be2aca5de0b05a8b'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:24:13.147040 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 818/1651] 10.222.0.15 () {58 vars in 1224 bytes} [Thu Sep 15 12:24:12 2022] DELETE /identity/v3/users/934491e7f02a4f14be2aca5de0b05a8b => generated 0 bytes in 332 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:24:14.040014 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc83a1c-1eb4-4051-9cda-cf8df62dad6e admin admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:24:14.044628 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-fbc83a1c-1eb4-4051-9cda-cf8df62dad6e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:24:14.076182 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc83a1c-1eb4-4051-9cda-cf8df62dad6e admin admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:24:14.198364 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650efaac-85ea-4d3b-8036-00f87ca4d8b5 admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:24:14.206501 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-650efaac-85ea-4d3b-8036-00f87ca4d8b5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:24:14.245990 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7000f86-1520-484c-a855-6703c10854ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:24:14.246747 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7000f86-1520-484c-a855-6703c10854ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:14.247816 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7000f86-1520-484c-a855-6703c10854ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:14.248219 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7000f86-1520-484c-a855-6703c10854ab admin admin] PATH_INFO: `/v3/projects/eece3bf66bcb4765b9e94a2f1a0ed1e5` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:14.249424 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7000f86-1520-484c-a855-6703c10854ab admin admin] RBAC: Authorizing `identity:delete_project(project_id=eece3bf66bcb4765b9e94a2f1a0ed1e5)` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:24:14.251549 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7000f86-1520-484c-a855-6703c10854ab admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:24:14.370480 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650efaac-85ea-4d3b-8036-00f87ca4d8b5 admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:24:14.387589 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-b7000f86-1520-484c-a855-6703c10854ab admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eece3bf66bcb4765b9e94a2f1a0ed1e5', 'request_id': 'req-b7000f86-1520-484c-a855-6703c10854ab'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:24:14.401473 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-b7000f86-1520-484c-a855-6703c10854ab admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eece3bf66bcb4765b9e94a2f1a0ed1e5', 'request_id': 'req-b7000f86-1520-484c-a855-6703c10854ab'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:24:14.412271 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-b7000f86-1520-484c-a855-6703c10854ab admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eece3bf66bcb4765b9e94a2f1a0ed1e5', 'request_id': 'req-b7000f86-1520-484c-a855-6703c10854ab'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:24:14.444847 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-b7000f86-1520-484c-a855-6703c10854ab admin admin] The token cache is being invalidate because project eece3bf66bcb4765b9e94a2f1a0ed1e5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81449) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 12:24:14.445106 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-b7000f86-1520-484c-a855-6703c10854ab admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:24:14.446975 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-b7000f86-1520-484c-a855-6703c10854ab admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:24:14.450079 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 834/1652] 10.222.0.15 () {58 vars in 1233 bytes} [Thu Sep 15 12:24:14 2022] DELETE /identity/v3/projects/eece3bf66bcb4765b9e94a2f1a0ed1e5 => generated 0 bytes in 410 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:24:14.460171 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db87c63b-adc3-4d72-8a70-bfc9fc090611 None None] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:14.460622 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db87c63b-adc3-4d72-8a70-bfc9fc090611 None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:14.461066 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db87c63b-adc3-4d72-8a70-bfc9fc090611 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:14.531118 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.auth.core [None req-db87c63b-adc3-4d72-8a70-bfc9fc090611 None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81449) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:24:14.532460 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-db87c63b-adc3-4d72-8a70-bfc9fc090611 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:24:14.574447 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:24:14.592124 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d07a9c9-5e9e-4a2e-bd35-abbb6e6a7618 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:24:14.592881 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d07a9c9-5e9e-4a2e-bd35-abbb6e6a7618 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:14.593107 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d07a9c9-5e9e-4a2e-bd35-abbb6e6a7618 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:14.593381 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d07a9c9-5e9e-4a2e-bd35-abbb6e6a7618 admin admin] PATH_INFO: `/v3/projects/df56dce262544960bbba4f9308f98fc2` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:14.598172 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d07a9c9-5e9e-4a2e-bd35-abbb6e6a7618 admin admin] RBAC: Authorizing `identity:delete_project(project_id=df56dce262544960bbba4f9308f98fc2)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:24:14.600142 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d07a9c9-5e9e-4a2e-bd35-abbb6e6a7618 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:24:14.645577 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:24:14.766721 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 835/1653] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 12:24:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:24:14.774667 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af007675-b2aa-46b1-acc6-8c8c8fe2757b None None] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:14.774667 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af007675-b2aa-46b1-acc6-8c8c8fe2757b None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:14.774667 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af007675-b2aa-46b1-acc6-8c8c8fe2757b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:14.779620 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-0d07a9c9-5e9e-4a2e-bd35-abbb6e6a7618 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'df56dce262544960bbba4f9308f98fc2', 'request_id': 'req-0d07a9c9-5e9e-4a2e-bd35-abbb6e6a7618'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:24:14.793322 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-0d07a9c9-5e9e-4a2e-bd35-abbb6e6a7618 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'df56dce262544960bbba4f9308f98fc2', 'request_id': 'req-0d07a9c9-5e9e-4a2e-bd35-abbb6e6a7618'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:24:14.808920 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-0d07a9c9-5e9e-4a2e-bd35-abbb6e6a7618 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'df56dce262544960bbba4f9308f98fc2', 'request_id': 'req-0d07a9c9-5e9e-4a2e-bd35-abbb6e6a7618'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:24:14.828591 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.auth.core [None req-af007675-b2aa-46b1-acc6-8c8c8fe2757b None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81449) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:24:14.833728 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-af007675-b2aa-46b1-acc6-8c8c8fe2757b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:24:14.847879 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-0d07a9c9-5e9e-4a2e-bd35-abbb6e6a7618 admin admin] The token cache is being invalidate because project df56dce262544960bbba4f9308f98fc2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81448) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 12:24:14.848613 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-0d07a9c9-5e9e-4a2e-bd35-abbb6e6a7618 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:24:14.851317 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-0d07a9c9-5e9e-4a2e-bd35-abbb6e6a7618 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:24:14.854544 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 819/1654] 10.222.0.15 () {58 vars in 1233 bytes} [Thu Sep 15 12:24:14 2022] DELETE /identity/v3/projects/df56dce262544960bbba4f9308f98fc2 => generated 0 bytes in 657 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:24:14.859687 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d07a9c9-5e9e-4a2e-bd35-abbb6e6a7618 admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:24:14.863653 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-0d07a9c9-5e9e-4a2e-bd35-abbb6e6a7618 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:24:15.009922 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 836/1655] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 12:24:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 242 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:24:15.016093 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af007675-b2aa-46b1-acc6-8c8c8fe2757b None None] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:24:15.028077 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d07a9c9-5e9e-4a2e-bd35-abbb6e6a7618 admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:24:15.039698 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96ac220-f552-46a2-b085-c0389293766a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:24:15.040418 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96ac220-f552-46a2-b085-c0389293766a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:15.040701 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96ac220-f552-46a2-b085-c0389293766a admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:15.040998 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96ac220-f552-46a2-b085-c0389293766a admin admin] PATH_INFO: `/v3/users/abd7a183d2434dc8b864634cd84c9d4e` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:15.046385 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96ac220-f552-46a2-b085-c0389293766a admin admin] RBAC: Authorizing `identity:delete_user(user_id=abd7a183d2434dc8b864634cd84c9d4e)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:24:15.049241 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af007675-b2aa-46b1-acc6-8c8c8fe2757b None None] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:24:15.056313 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8272f69-6f0a-4d4b-bfc4-8cd34cdf0abb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:24:15.057714 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8272f69-6f0a-4d4b-bfc4-8cd34cdf0abb admin admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:15.058154 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8272f69-6f0a-4d4b-bfc4-8cd34cdf0abb admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:15.058577 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8272f69-6f0a-4d4b-bfc4-8cd34cdf0abb admin admin] PATH_INFO: `/v3/domains` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:15.059429 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8272f69-6f0a-4d4b-bfc4-8cd34cdf0abb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81449) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:24:15.059647 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96ac220-f552-46a2-b085-c0389293766a admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:24:15.060464 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8272f69-6f0a-4d4b-bfc4-8cd34cdf0abb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:24:15.075065 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8272f69-6f0a-4d4b-bfc4-8cd34cdf0abb admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:24:15.130031 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 837/1656] 10.222.0.15 () {58 vars in 1153 bytes} [Thu Sep 15 12:24:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:24:15.136697 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8272f69-6f0a-4d4b-bfc4-8cd34cdf0abb admin admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:24:15.155291 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8272f69-6f0a-4d4b-bfc4-8cd34cdf0abb admin admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:24:15.207486 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861a8c01-985a-4802-a5e8-7c7511cadf41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:24:15.208674 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861a8c01-985a-4802-a5e8-7c7511cadf41 admin admin] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:15.209497 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861a8c01-985a-4802-a5e8-7c7511cadf41 admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:15.210845 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861a8c01-985a-4802-a5e8-7c7511cadf41 admin admin] PATH_INFO: `/v3/projects` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:15.211751 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861a8c01-985a-4802-a5e8-7c7511cadf41 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:24:15.224214 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861a8c01-985a-4802-a5e8-7c7511cadf41 admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:24:15.231332 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-e96ac220-f552-46a2-b085-c0389293766a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'abd7a183d2434dc8b864634cd84c9d4e'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:24:15.271832 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-e96ac220-f552-46a2-b085-c0389293766a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'abd7a183d2434dc8b864634cd84c9d4e'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:24:15.312379 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-e96ac220-f552-46a2-b085-c0389293766a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'abd7a183d2434dc8b864634cd84c9d4e'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:24:15.313397 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-e96ac220-f552-46a2-b085-c0389293766a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'abd7a183d2434dc8b864634cd84c9d4e'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:24:15.331349 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 838/1657] 10.222.0.15 () {60 vars in 1153 bytes} [Thu Sep 15 12:24:15 2022] POST /identity/v3/projects => generated 385 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 12:24:15.337789 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861a8c01-985a-4802-a5e8-7c7511cadf41 admin admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:24:15.340502 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-e96ac220-f552-46a2-b085-c0389293766a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'abd7a183d2434dc8b864634cd84c9d4e'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:24:15.343870 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-861a8c01-985a-4802-a5e8-7c7511cadf41 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:24:15.361495 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 820/1658] 10.222.0.15 () {58 vars in 1224 bytes} [Thu Sep 15 12:24:14 2022] DELETE /identity/v3/users/abd7a183d2434dc8b864634cd84c9d4e => generated 0 bytes in 503 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:24:15.446057 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861a8c01-985a-4802-a5e8-7c7511cadf41 admin admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:24:15.493456 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:24:15.575460 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5980bf-3f08-4f53-a8ef-74c3b7e58214 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:24:15.578067 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5980bf-3f08-4f53-a8ef-74c3b7e58214 admin admin] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:15.578586 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5980bf-3f08-4f53-a8ef-74c3b7e58214 admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:15.579784 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5980bf-3f08-4f53-a8ef-74c3b7e58214 admin admin] PATH_INFO: `/v3/users` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:15.581095 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5980bf-3f08-4f53-a8ef-74c3b7e58214 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:24:15.583376 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca5980bf-3f08-4f53-a8ef-74c3b7e58214 admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:24:15.621132 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 839/1659] 10.222.0.15 () {60 vars in 1144 bytes} [Thu Sep 15 12:24:15 2022] POST /identity/v3/users => generated 293 bytes in 284 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 12:24:15.625654 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96ac220-f552-46a2-b085-c0389293766a admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:24:15.640505 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96ac220-f552-46a2-b085-c0389293766a admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:24:15.650853 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff155873-0e3f-4d04-914f-cace1e7d2cf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:24:15.651969 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff155873-0e3f-4d04-914f-cace1e7d2cf9 admin admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:15.652805 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff155873-0e3f-4d04-914f-cace1e7d2cf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:15.653603 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff155873-0e3f-4d04-914f-cace1e7d2cf9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:15.654807 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff155873-0e3f-4d04-914f-cace1e7d2cf9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:24:15.657289 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff155873-0e3f-4d04-914f-cace1e7d2cf9 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:24:15.678343 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 821/1660] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 12:24:15 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 12:24:15.684328 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5980bf-3f08-4f53-a8ef-74c3b7e58214 admin admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:24:15.703117 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5980bf-3f08-4f53-a8ef-74c3b7e58214 admin admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:24:15.709379 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e086fb40-9694-4442-95d0-3b17ca0fa80c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:24:15.710360 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e086fb40-9694-4442-95d0-3b17ca0fa80c admin admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:15.710857 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e086fb40-9694-4442-95d0-3b17ca0fa80c admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:15.711752 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e086fb40-9694-4442-95d0-3b17ca0fa80c admin admin] PATH_INFO: `/v3/roles` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:15.717633 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e086fb40-9694-4442-95d0-3b17ca0fa80c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:24:15.719724 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e086fb40-9694-4442-95d0-3b17ca0fa80c admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:24:15.740037 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 840/1661] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 12:24:15 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 12:24:15.743921 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff155873-0e3f-4d04-914f-cace1e7d2cf9 admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:24:15.761823 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff155873-0e3f-4d04-914f-cace1e7d2cf9 admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:24:15.771678 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e369be-e5b8-4255-b472-410e808057bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:24:15.772773 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e369be-e5b8-4255-b472-410e808057bc admin admin] REQUEST_METHOD: `PUT` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:15.773205 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e369be-e5b8-4255-b472-410e808057bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:15.773621 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e369be-e5b8-4255-b472-410e808057bc admin admin] PATH_INFO: `/v3/projects/f832edf73ffb484586c4c4db9544cc26/users/54576808c2824d07a5bcfcd52cb11720/roles/0679d6628a25438d90dc4bdc0401eda2` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:15.776510 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:24:15.800798 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e369be-e5b8-4255-b472-410e808057bc admin admin] RBAC: Authorizing `identity:create_grant(project_id=f832edf73ffb484586c4c4db9544cc26, user_id=54576808c2824d07a5bcfcd52cb11720, role_id=0679d6628a25438d90dc4bdc0401eda2)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:24:15.803882 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e369be-e5b8-4255-b472-410e808057bc admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:24:15.833479 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cea9a1-724f-4e33-913c-f6d7befce74a None None] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:15.833768 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cea9a1-724f-4e33-913c-f6d7befce74a None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:15.834040 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cea9a1-724f-4e33-913c-f6d7befce74a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:15.837374 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 822/1662] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 15 12:24:15 2022] PUT /identity/v3/projects/f832edf73ffb484586c4c4db9544cc26/users/54576808c2824d07a5bcfcd52cb11720/roles/0679d6628a25438d90dc4bdc0401eda2 => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:24:15.880706 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.auth.core [None req-b9cea9a1-724f-4e33-913c-f6d7befce74a None None] MFA Rules not processed for user `54576808c2824d07a5bcfcd52cb11720`. Rule list: `[]` (Enabled: `True`). {{(pid=81449) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:24:15.882848 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-b9cea9a1-724f-4e33-913c-f6d7befce74a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:24:15.992829 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 841/1663] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 12:24:15 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:24:15.997540 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e369be-e5b8-4255-b472-410e808057bc admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:24:16.216874 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e369be-e5b8-4255-b472-410e808057bc admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:24:16.223935 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2b2314-9f8c-4ec1-a75d-9bf4d8ad0d4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:24:16.225170 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2b2314-9f8c-4ec1-a75d-9bf4d8ad0d4a admin admin] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:16.225523 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2b2314-9f8c-4ec1-a75d-9bf4d8ad0d4a admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:16.225863 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2b2314-9f8c-4ec1-a75d-9bf4d8ad0d4a admin admin] PATH_INFO: `/v3/projects` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:16.226580 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2b2314-9f8c-4ec1-a75d-9bf4d8ad0d4a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:24:16.228571 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2b2314-9f8c-4ec1-a75d-9bf4d8ad0d4a admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:24:16.278079 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 823/1664] 10.222.0.15 () {60 vars in 1153 bytes} [Thu Sep 15 12:24:15 2022] POST /identity/v3/projects => generated 387 bytes in 281 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 12:24:16.278754 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9cea9a1-724f-4e33-913c-f6d7befce74a None None] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:24:16.300572 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9cea9a1-724f-4e33-913c-f6d7befce74a None None] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:24:16.446582 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57973171-5078-4eb7-a71d-3320e8a08352 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:24:16.447210 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57973171-5078-4eb7-a71d-3320e8a08352 admin admin] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:16.447482 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57973171-5078-4eb7-a71d-3320e8a08352 admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:16.448299 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57973171-5078-4eb7-a71d-3320e8a08352 admin admin] PATH_INFO: `/v3/users` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:16.448996 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57973171-5078-4eb7-a71d-3320e8a08352 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:24:16.451515 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57973171-5078-4eb7-a71d-3320e8a08352 admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:24:16.492071 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2b2314-9f8c-4ec1-a75d-9bf4d8ad0d4a admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:24:16.503185 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 842/1665] 10.222.0.15 () {60 vars in 1144 bytes} [Thu Sep 15 12:24:16 2022] POST /identity/v3/users => generated 300 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 15 12:24:16.504788 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2b2314-9f8c-4ec1-a75d-9bf4d8ad0d4a admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:24:16.511521 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ccab5a-8d42-4d88-ad85-499c8fe77add admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:24:16.512171 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ccab5a-8d42-4d88-ad85-499c8fe77add admin admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:16.512451 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ccab5a-8d42-4d88-ad85-499c8fe77add admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:16.512719 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ccab5a-8d42-4d88-ad85-499c8fe77add admin admin] PATH_INFO: `/v3/roles` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:16.515292 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ccab5a-8d42-4d88-ad85-499c8fe77add admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:24:16.516938 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ccab5a-8d42-4d88-ad85-499c8fe77add admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:24:16.535446 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57973171-5078-4eb7-a71d-3320e8a08352 admin admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:24:16.547691 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 824/1666] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 12:24:16 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 12:24:16.552465 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57973171-5078-4eb7-a71d-3320e8a08352 admin admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:24:16.566398 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1a4220-121c-49d9-8fd7-bdb65dd3d5d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:24:16.568339 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1a4220-121c-49d9-8fd7-bdb65dd3d5d0 admin admin] REQUEST_METHOD: `PUT` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:16.573831 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1a4220-121c-49d9-8fd7-bdb65dd3d5d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:16.574523 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1a4220-121c-49d9-8fd7-bdb65dd3d5d0 admin admin] PATH_INFO: `/v3/projects/e009b017ce8c40a098d047fdc98b06cd/users/41db01ef23f647308a2fbe90526f36f9/roles/9c769b5c82884766b2c510b1f0f1364c` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:16.576886 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ccab5a-8d42-4d88-ad85-499c8fe77add admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:24:16.578918 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-35ccab5a-8d42-4d88-ad85-499c8fe77add admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:24:16.609004 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ccab5a-8d42-4d88-ad85-499c8fe77add admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:24:16.621065 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79ade60-82b8-47ab-82ad-e89ee6cbfd18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:24:16.622549 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1a4220-121c-49d9-8fd7-bdb65dd3d5d0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e009b017ce8c40a098d047fdc98b06cd, user_id=41db01ef23f647308a2fbe90526f36f9, role_id=9c769b5c82884766b2c510b1f0f1364c)` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:24:16.623903 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79ade60-82b8-47ab-82ad-e89ee6cbfd18 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:16.624198 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79ade60-82b8-47ab-82ad-e89ee6cbfd18 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:16.624469 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79ade60-82b8-47ab-82ad-e89ee6cbfd18 admin admin] PATH_INFO: `/v3/projects/b16dfc33fb784887b773d73227fe99b2` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:16.627472 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1a4220-121c-49d9-8fd7-bdb65dd3d5d0 admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:24:16.630817 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79ade60-82b8-47ab-82ad-e89ee6cbfd18 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b16dfc33fb784887b773d73227fe99b2)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:24:16.634708 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79ade60-82b8-47ab-82ad-e89ee6cbfd18 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:24:16.665257 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 843/1667] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 15 12:24:16 2022] PUT /identity/v3/projects/e009b017ce8c40a098d047fdc98b06cd/users/41db01ef23f647308a2fbe90526f36f9/roles/9c769b5c82884766b2c510b1f0f1364c => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:24:16.668545 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1a4220-121c-49d9-8fd7-bdb65dd3d5d0 admin admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:24:16.682856 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:24:16.820457 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-b79ade60-82b8-47ab-82ad-e89ee6cbfd18 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b16dfc33fb784887b773d73227fe99b2', 'request_id': 'req-b79ade60-82b8-47ab-82ad-e89ee6cbfd18'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:24:16.852795 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-b79ade60-82b8-47ab-82ad-e89ee6cbfd18 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b16dfc33fb784887b773d73227fe99b2', 'request_id': 'req-b79ade60-82b8-47ab-82ad-e89ee6cbfd18'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:24:16.873330 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-b79ade60-82b8-47ab-82ad-e89ee6cbfd18 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b16dfc33fb784887b773d73227fe99b2', 'request_id': 'req-b79ade60-82b8-47ab-82ad-e89ee6cbfd18'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:24:16.913066 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-b79ade60-82b8-47ab-82ad-e89ee6cbfd18 admin admin] The token cache is being invalidate because project b16dfc33fb784887b773d73227fe99b2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81448) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 12:24:16.918464 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-b79ade60-82b8-47ab-82ad-e89ee6cbfd18 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:24:16.920483 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-b79ade60-82b8-47ab-82ad-e89ee6cbfd18 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:24:16.927786 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 825/1668] 10.222.0.15 () {58 vars in 1233 bytes} [Thu Sep 15 12:24:16 2022] DELETE /identity/v3/projects/b16dfc33fb784887b773d73227fe99b2 => generated 0 bytes in 352 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:24:16.929578 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1a4220-121c-49d9-8fd7-bdb65dd3d5d0 admin admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:24:16.935123 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839ef73f-8627-4bad-a7d4-2431a00a14d3 None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:16.935366 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839ef73f-8627-4bad-a7d4-2431a00a14d3 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:16.935642 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839ef73f-8627-4bad-a7d4-2431a00a14d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:17.011329 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-839ef73f-8627-4bad-a7d4-2431a00a14d3 None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:24:17.012566 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-839ef73f-8627-4bad-a7d4-2431a00a14d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:24:17.081127 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6632e0-feb7-4f55-af3b-13a46ea6ebda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:24:17.084681 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6632e0-feb7-4f55-af3b-13a46ea6ebda admin admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:17.085006 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6632e0-feb7-4f55-af3b-13a46ea6ebda admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:17.085277 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6632e0-feb7-4f55-af3b-13a46ea6ebda admin admin] PATH_INFO: `/v3/roles` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:17.085960 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6632e0-feb7-4f55-af3b-13a46ea6ebda admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:24:17.090335 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6632e0-feb7-4f55-af3b-13a46ea6ebda admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:24:17.109467 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 844/1669] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 12:24:16 2022] GET /identity/v3/roles => generated 1420 bytes in 444 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 12:24:17.117469 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6632e0-feb7-4f55-af3b-13a46ea6ebda admin admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:24:17.124509 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-eb6632e0-feb7-4f55-af3b-13a46ea6ebda admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:24:17.136323 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:24:17.164404 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6632e0-feb7-4f55-af3b-13a46ea6ebda admin admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:24:17.177506 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac6cb356-0504-4815-adfc-e22f4eb1959e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:24:17.179651 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6cb356-0504-4815-adfc-e22f4eb1959e admin admin] REQUEST_METHOD: `PUT` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:17.179946 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6cb356-0504-4815-adfc-e22f4eb1959e admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:17.180402 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6cb356-0504-4815-adfc-e22f4eb1959e admin admin] PATH_INFO: `/v3/projects/e009b017ce8c40a098d047fdc98b06cd/users/41db01ef23f647308a2fbe90526f36f9/roles/9c769b5c82884766b2c510b1f0f1364c` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:17.201341 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6cb356-0504-4815-adfc-e22f4eb1959e admin admin] RBAC: Authorizing `identity:create_grant(project_id=e009b017ce8c40a098d047fdc98b06cd, user_id=41db01ef23f647308a2fbe90526f36f9, role_id=9c769b5c82884766b2c510b1f0f1364c)` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:24:17.209968 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac6cb356-0504-4815-adfc-e22f4eb1959e admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:24:17.230274 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 826/1670] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 12:24:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 298 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:24:17.239325 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a89541a-aac9-43c4-9283-53580546d033 None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:17.239602 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a89541a-aac9-43c4-9283-53580546d033 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:17.239878 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a89541a-aac9-43c4-9283-53580546d033 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:17.246124 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 845/1671] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 15 12:24:17 2022] PUT /identity/v3/projects/e009b017ce8c40a098d047fdc98b06cd/users/41db01ef23f647308a2fbe90526f36f9/roles/9c769b5c82884766b2c510b1f0f1364c => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:24:17.269048 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfdb201-07a1-4d3e-9f65-4549c824ee37 None None] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:17.269336 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfdb201-07a1-4d3e-9f65-4549c824ee37 None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:17.269616 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfdb201-07a1-4d3e-9f65-4549c824ee37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:17.301477 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-7a89541a-aac9-43c4-9283-53580546d033 None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:24:17.304711 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-7a89541a-aac9-43c4-9283-53580546d033 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:24:17.332800 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.auth.core [None req-ebfdb201-07a1-4d3e-9f65-4549c824ee37 None None] MFA Rules not processed for user `41db01ef23f647308a2fbe90526f36f9`. Rule list: `[]` (Enabled: `True`). {{(pid=81449) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:24:17.334034 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-ebfdb201-07a1-4d3e-9f65-4549c824ee37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:24:17.454824 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 827/1672] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 12:24:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:24:17.459451 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a89541a-aac9-43c4-9283-53580546d033 None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:24:17.503411 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a89541a-aac9-43c4-9283-53580546d033 None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:24:17.517444 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9c9dcf-ba8d-4f68-ac5a-537c7ff43c9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:24:17.519586 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9c9dcf-ba8d-4f68-ac5a-537c7ff43c9f admin admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:17.519879 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9c9dcf-ba8d-4f68-ac5a-537c7ff43c9f admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:17.520417 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9c9dcf-ba8d-4f68-ac5a-537c7ff43c9f admin admin] PATH_INFO: `/v3/domains` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:17.522037 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9c9dcf-ba8d-4f68-ac5a-537c7ff43c9f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:24:17.523043 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9c9dcf-ba8d-4f68-ac5a-537c7ff43c9f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:24:17.525567 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9c9dcf-ba8d-4f68-ac5a-537c7ff43c9f admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:24:17.557564 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 846/1673] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 12:24:17 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 288 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:24:17.562808 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 828/1674] 10.222.0.15 () {58 vars in 1153 bytes} [Thu Sep 15 12:24:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:24:17.564557 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebfdb201-07a1-4d3e-9f65-4549c824ee37 None None] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:24:17.592115 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebfdb201-07a1-4d3e-9f65-4549c824ee37 None None] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:24:17.608789 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969fa61a-6ef0-4601-abc1-4429445001ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:24:17.609615 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969fa61a-6ef0-4601-abc1-4429445001ac admin admin] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:17.612130 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969fa61a-6ef0-4601-abc1-4429445001ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:17.612509 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969fa61a-6ef0-4601-abc1-4429445001ac admin admin] PATH_INFO: `/v3/projects` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:17.613159 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-969fa61a-6ef0-4601-abc1-4429445001ac admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:24:17.615324 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-969fa61a-6ef0-4601-abc1-4429445001ac admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:24:17.637059 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9c9dcf-ba8d-4f68-ac5a-537c7ff43c9f admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:24:17.638895 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-6e9c9dcf-ba8d-4f68-ac5a-537c7ff43c9f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:24:17.687383 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:24:17.725469 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 847/1675] 10.222.0.15 () {60 vars in 1153 bytes} [Thu Sep 15 12:24:17 2022] POST /identity/v3/projects => generated 365 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 12:24:17.727446 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969fa61a-6ef0-4601-abc1-4429445001ac admin admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:24:17.836215 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9c9dcf-ba8d-4f68-ac5a-537c7ff43c9f admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:24:17.905943 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969fa61a-6ef0-4601-abc1-4429445001ac admin admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:24:17.927267 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f407114-9efd-4e5c-ba7a-15ae01ff70c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:24:17.933334 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f407114-9efd-4e5c-ba7a-15ae01ff70c8 admin admin] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:17.933893 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f407114-9efd-4e5c-ba7a-15ae01ff70c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:17.934660 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f407114-9efd-4e5c-ba7a-15ae01ff70c8 admin admin] PATH_INFO: `/v3/users` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:17.942967 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f407114-9efd-4e5c-ba7a-15ae01ff70c8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:24:17.962901 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e176eeb-b3db-4501-bfd1-55fc720d26d0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8191d3ccb874529b32f8d0026db629d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '540a32ef88764e8c8d9201ae22f6ddaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:24:17.964142 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e176eeb-b3db-4501-bfd1-55fc720d26d0 service cinder] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:17.964429 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e176eeb-b3db-4501-bfd1-55fc720d26d0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:17.964701 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e176eeb-b3db-4501-bfd1-55fc720d26d0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:17.971572 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f407114-9efd-4e5c-ba7a-15ae01ff70c8 admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:24:18.062290 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 848/1676] 10.222.0.15 () {60 vars in 1144 bytes} [Thu Sep 15 12:24:17 2022] POST /identity/v3/users => generated 283 bytes in 335 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 12:24:18.064082 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f407114-9efd-4e5c-ba7a-15ae01ff70c8 admin admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:24:18.079393 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:24:18.080144 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f407114-9efd-4e5c-ba7a-15ae01ff70c8 admin admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:24:18.090476 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d193e99b-a470-4f21-8920-9ca943dcdc04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:24:18.091504 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d193e99b-a470-4f21-8920-9ca943dcdc04 admin admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:18.092736 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d193e99b-a470-4f21-8920-9ca943dcdc04 admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:18.094265 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d193e99b-a470-4f21-8920-9ca943dcdc04 admin admin] PATH_INFO: `/v3/roles` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:18.095267 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d193e99b-a470-4f21-8920-9ca943dcdc04 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:24:18.098904 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d193e99b-a470-4f21-8920-9ca943dcdc04 admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:24:18.116427 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 849/1677] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 12:24:18 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 15 12:24:18.119050 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d193e99b-a470-4f21-8920-9ca943dcdc04 admin admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:24:18.121527 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e176eeb-b3db-4501-bfd1-55fc720d26d0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:24:18.122622 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e176eeb-b3db-4501-bfd1-55fc720d26d0 service cinder] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:24:18.137664 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 829/1678] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 12:24:17 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 501 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 12:24:18.159628 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d193e99b-a470-4f21-8920-9ca943dcdc04 admin admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:24:18.165585 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ebab56-54d7-4cb4-b90c-be7ff46631d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:24:18.166653 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ebab56-54d7-4cb4-b90c-be7ff46631d2 admin admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:18.167333 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ebab56-54d7-4cb4-b90c-be7ff46631d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:18.167832 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ebab56-54d7-4cb4-b90c-be7ff46631d2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:18.168927 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ebab56-54d7-4cb4-b90c-be7ff46631d2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:24:18.171082 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09ebab56-54d7-4cb4-b90c-be7ff46631d2 admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:24:18.206780 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e176eeb-b3db-4501-bfd1-55fc720d26d0 service cinder] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:24:18.209040 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 850/1679] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 12:24:18 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 12:24:18.224638 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e176eeb-b3db-4501-bfd1-55fc720d26d0 service cinder] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:24:18.230808 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3103707-54a0-4464-bad9-503bb358585f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:24:18.231897 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3103707-54a0-4464-bad9-503bb358585f admin admin] REQUEST_METHOD: `PUT` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:18.232102 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3103707-54a0-4464-bad9-503bb358585f admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:18.232393 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3103707-54a0-4464-bad9-503bb358585f admin admin] PATH_INFO: `/v3/projects/b657bd5a9cfc4f7bb7130356dcf7966b/users/6b2c7e7a7d514af89e9c1232e1672445/roles/0679d6628a25438d90dc4bdc0401eda2` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:18.268788 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ebab56-54d7-4cb4-b90c-be7ff46631d2 admin admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:24:18.269680 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3103707-54a0-4464-bad9-503bb358585f admin admin] RBAC: Authorizing `identity:create_grant(project_id=b657bd5a9cfc4f7bb7130356dcf7966b, user_id=6b2c7e7a7d514af89e9c1232e1672445, role_id=0679d6628a25438d90dc4bdc0401eda2)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:24:18.273552 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3103707-54a0-4464-bad9-503bb358585f admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:24:18.289512 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ebab56-54d7-4cb4-b90c-be7ff46631d2 admin admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:24:18.301323 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36bc13bc-87bb-41e1-b46a-d99fb2759d62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:24:18.302280 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bc13bc-87bb-41e1-b46a-d99fb2759d62 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:18.304059 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bc13bc-87bb-41e1-b46a-d99fb2759d62 admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:18.304668 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36bc13bc-87bb-41e1-b46a-d99fb2759d62 admin admin] PATH_INFO: `/v3/users/54576808c2824d07a5bcfcd52cb11720` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:18.307986 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36bc13bc-87bb-41e1-b46a-d99fb2759d62 admin admin] RBAC: Authorizing `identity:delete_user(user_id=54576808c2824d07a5bcfcd52cb11720)` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:24:18.313342 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36bc13bc-87bb-41e1-b46a-d99fb2759d62 admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:24:18.317599 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 830/1680] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 15 12:24:18 2022] PUT /identity/v3/projects/b657bd5a9cfc4f7bb7130356dcf7966b/users/6b2c7e7a7d514af89e9c1232e1672445/roles/0679d6628a25438d90dc4bdc0401eda2 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 12:24:18.330216 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01934b09-bc29-4e50-8444-e439d17cc505 None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:18.330777 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01934b09-bc29-4e50-8444-e439d17cc505 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:18.331300 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01934b09-bc29-4e50-8444-e439d17cc505 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:18.394650 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-01934b09-bc29-4e50-8444-e439d17cc505 None None] MFA Rules not processed for user `6b2c7e7a7d514af89e9c1232e1672445`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:24:18.396048 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-01934b09-bc29-4e50-8444-e439d17cc505 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:24:18.455230 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-36bc13bc-87bb-41e1-b46a-d99fb2759d62 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '54576808c2824d07a5bcfcd52cb11720'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:24:18.477479 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-36bc13bc-87bb-41e1-b46a-d99fb2759d62 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '54576808c2824d07a5bcfcd52cb11720'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:24:18.505031 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-36bc13bc-87bb-41e1-b46a-d99fb2759d62 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '54576808c2824d07a5bcfcd52cb11720'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:24:18.506969 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-36bc13bc-87bb-41e1-b46a-d99fb2759d62 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '54576808c2824d07a5bcfcd52cb11720'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:24:18.527715 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-36bc13bc-87bb-41e1-b46a-d99fb2759d62 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '54576808c2824d07a5bcfcd52cb11720'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:24:18.547492 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 831/1682] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 12:24:18 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 12:24:18.550974 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 851/1682] 10.222.0.15 () {58 vars in 1224 bytes} [Thu Sep 15 12:24:18 2022] DELETE /identity/v3/users/54576808c2824d07a5bcfcd52cb11720 => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 12:24:18.551237 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08b521b-ecdf-49ce-a203-a2d87d23ced9 None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:18.551470 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08b521b-ecdf-49ce-a203-a2d87d23ced9 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:18.551855 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08b521b-ecdf-49ce-a203-a2d87d23ced9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:18.552111 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36bc13bc-87bb-41e1-b46a-d99fb2759d62 admin admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:24:18.556517 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-36bc13bc-87bb-41e1-b46a-d99fb2759d62 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:24:18.593019 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-b08b521b-ecdf-49ce-a203-a2d87d23ced9 None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:24:18.594450 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-b08b521b-ecdf-49ce-a203-a2d87d23ced9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:24:18.668911 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:24:18.722606 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36bc13bc-87bb-41e1-b46a-d99fb2759d62 admin admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:24:18.734194 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53e8c0a-55c9-4b77-b889-fb3e1a9db027 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:24:18.734832 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53e8c0a-55c9-4b77-b889-fb3e1a9db027 admin admin] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:18.735109 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53e8c0a-55c9-4b77-b889-fb3e1a9db027 admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:18.735376 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53e8c0a-55c9-4b77-b889-fb3e1a9db027 admin admin] PATH_INFO: `/v3/projects` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:18.736020 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53e8c0a-55c9-4b77-b889-fb3e1a9db027 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:24:18.743326 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53e8c0a-55c9-4b77-b889-fb3e1a9db027 admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:24:18.748332 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 832/1683] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 12:24:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 12:24:18.749788 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83585061-e51d-41c7-b699-7abadba13fc3 None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:18.752675 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83585061-e51d-41c7-b699-7abadba13fc3 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:18.753246 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83585061-e51d-41c7-b699-7abadba13fc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:18.802966 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 852/1684] 10.222.0.15 () {60 vars in 1153 bytes} [Thu Sep 15 12:24:18 2022] POST /identity/v3/projects => generated 363 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 12:24:18.803860 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53e8c0a-55c9-4b77-b889-fb3e1a9db027 admin admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:24:18.811357 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-f53e8c0a-55c9-4b77-b889-fb3e1a9db027 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:24:18.833623 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-83585061-e51d-41c7-b699-7abadba13fc3 None None] MFA Rules not processed for user `b5c77cfb2ad8404f9d0dff4f9bdb98ba`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:24:18.836061 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-83585061-e51d-41c7-b699-7abadba13fc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:24:18.901440 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:24:18.937180 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53e8c0a-55c9-4b77-b889-fb3e1a9db027 admin admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:24:18.945420 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3182600d-7923-4f32-b768-b554cc433cd0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfbb4d6809064f9cb9e06626ed7d6814', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '540a32ef88764e8c8d9201ae22f6ddaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:24:18.946094 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3182600d-7923-4f32-b768-b554cc433cd0 service neutron] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:18.946316 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3182600d-7923-4f32-b768-b554cc433cd0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:18.946587 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3182600d-7923-4f32-b768-b554cc433cd0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:18.984802 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 833/1685] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 12:24:18 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:24:18.984802 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83585061-e51d-41c7-b699-7abadba13fc3 None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:24:19.083565 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3182600d-7923-4f32-b768-b554cc433cd0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:24:19.085046 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3182600d-7923-4f32-b768-b554cc433cd0 service neutron] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:24:19.128658 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 853/1686] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 12:24:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 12:24:19.132484 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da869345-3ccd-4cb3-8b00-7bfc8cfc4eca None None] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:19.135878 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da869345-3ccd-4cb3-8b00-7bfc8cfc4eca None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:19.137261 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83585061-e51d-41c7-b699-7abadba13fc3 None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:24:19.138475 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da869345-3ccd-4cb3-8b00-7bfc8cfc4eca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:19.151964 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3590aeb-4d0c-4572-a0d5-8a735ed0f5e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:24:19.153815 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3590aeb-4d0c-4572-a0d5-8a735ed0f5e9 admin admin] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:19.154379 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3590aeb-4d0c-4572-a0d5-8a735ed0f5e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:19.165590 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3590aeb-4d0c-4572-a0d5-8a735ed0f5e9 admin admin] PATH_INFO: `/v3/users` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:19.169225 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3590aeb-4d0c-4572-a0d5-8a735ed0f5e9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:24:19.171278 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3590aeb-4d0c-4572-a0d5-8a735ed0f5e9 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:24:19.212600 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.auth.core [None req-da869345-3ccd-4cb3-8b00-7bfc8cfc4eca None None] MFA Rules not processed for user `b5c77cfb2ad8404f9d0dff4f9bdb98ba`. Rule list: `[]` (Enabled: `True`). {{(pid=81449) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:24:19.213911 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-da869345-3ccd-4cb3-8b00-7bfc8cfc4eca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:24:19.229652 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 854/1687] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 12:24:19 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:24:19.254960 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da869345-3ccd-4cb3-8b00-7bfc8cfc4eca None None] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:24:19.265447 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-da869345-3ccd-4cb3-8b00-7bfc8cfc4eca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:24:19.265891 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 834/1688] 10.222.0.15 () {60 vars in 1144 bytes} [Thu Sep 15 12:24:18 2022] POST /identity/v3/users => generated 288 bytes in 291 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 12:24:19.268394 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3590aeb-4d0c-4572-a0d5-8a735ed0f5e9 admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:24:19.299095 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3590aeb-4d0c-4572-a0d5-8a735ed0f5e9 admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:24:19.310330 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2900cc9-c604-469c-a400-92664769ee3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:24:19.311869 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2900cc9-c604-469c-a400-92664769ee3c admin admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:19.312383 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2900cc9-c604-469c-a400-92664769ee3c admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:19.313455 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2900cc9-c604-469c-a400-92664769ee3c admin admin] PATH_INFO: `/v3/roles` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:19.315551 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2900cc9-c604-469c-a400-92664769ee3c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:24:19.332269 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2900cc9-c604-469c-a400-92664769ee3c admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:24:19.374932 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 835/1689] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 12:24:19 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 12:24:19.377634 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2900cc9-c604-469c-a400-92664769ee3c admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:24:19.380548 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-d2900cc9-c604-469c-a400-92664769ee3c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:24:19.409678 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2900cc9-c604-469c-a400-92664769ee3c admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:24:19.428987 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44bd3360-23af-465c-942e-5d5dc2576616 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:24:19.428987 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bd3360-23af-465c-942e-5d5dc2576616 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:19.428987 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bd3360-23af-465c-942e-5d5dc2576616 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:19.428987 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bd3360-23af-465c-942e-5d5dc2576616 admin admin] PATH_INFO: `/v3/users/7b8262b6c3a04914a89ed3548af3bf7b` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:19.429567 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44bd3360-23af-465c-942e-5d5dc2576616 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7b8262b6c3a04914a89ed3548af3bf7b)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:24:19.429567 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44bd3360-23af-465c-942e-5d5dc2576616 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:24:19.476957 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:24:19.496909 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da869345-3ccd-4cb3-8b00-7bfc8cfc4eca None None] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:24:19.507822 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41eb0360-c839-4d6a-a19f-7f170337ce17 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eface89e261840a2abb4206be26455c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '540a32ef88764e8c8d9201ae22f6ddaf', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:24:19.508648 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41eb0360-c839-4d6a-a19f-7f170337ce17 service nova] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:19.508996 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41eb0360-c839-4d6a-a19f-7f170337ce17 service nova] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:19.509271 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41eb0360-c839-4d6a-a19f-7f170337ce17 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:19.526345 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41eb0360-c839-4d6a-a19f-7f170337ce17 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:24:19.541893 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41eb0360-c839-4d6a-a19f-7f170337ce17 service nova] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:24:19.551928 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-44bd3360-23af-465c-942e-5d5dc2576616 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7b8262b6c3a04914a89ed3548af3bf7b'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:24:19.578609 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 855/1690] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 12:24:19 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 12:24:19.580633 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41eb0360-c839-4d6a-a19f-7f170337ce17 service nova] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:24:19.598589 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-44bd3360-23af-465c-942e-5d5dc2576616 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7b8262b6c3a04914a89ed3548af3bf7b'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:24:19.628636 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41eb0360-c839-4d6a-a19f-7f170337ce17 service nova] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:24:19.636902 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-44bd3360-23af-465c-942e-5d5dc2576616 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7b8262b6c3a04914a89ed3548af3bf7b'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:24:19.637759 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-44bd3360-23af-465c-942e-5d5dc2576616 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7b8262b6c3a04914a89ed3548af3bf7b'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:24:19.653019 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5feb64-c166-446c-8af0-78f90afb01c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:24:19.653695 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5feb64-c166-446c-8af0-78f90afb01c6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:19.653973 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5feb64-c166-446c-8af0-78f90afb01c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:19.654245 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5feb64-c166-446c-8af0-78f90afb01c6 admin admin] PATH_INFO: `/v3/projects/19e6ec5e336d48f1a10ab8d91c95b1a1/users/68e37396900f49488ebc8b2a4459dea2/roles/9c769b5c82884766b2c510b1f0f1364c` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:19.667350 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-44bd3360-23af-465c-942e-5d5dc2576616 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7b8262b6c3a04914a89ed3548af3bf7b'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:24:19.691883 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 836/1691] 10.222.0.15 () {58 vars in 1224 bytes} [Thu Sep 15 12:24:19 2022] DELETE /identity/v3/users/7b8262b6c3a04914a89ed3548af3bf7b => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:24:19.697160 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18ae51d-9cd6-4522-a03b-283fb292f004 None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:19.697984 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18ae51d-9cd6-4522-a03b-283fb292f004 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:19.698260 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18ae51d-9cd6-4522-a03b-283fb292f004 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:19.712044 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5feb64-c166-446c-8af0-78f90afb01c6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=19e6ec5e336d48f1a10ab8d91c95b1a1, user_id=68e37396900f49488ebc8b2a4459dea2, role_id=9c769b5c82884766b2c510b1f0f1364c)` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:24:19.715303 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5feb64-c166-446c-8af0-78f90afb01c6 admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:24:19.737793 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-b18ae51d-9cd6-4522-a03b-283fb292f004 None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:24:19.739172 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-b18ae51d-9cd6-4522-a03b-283fb292f004 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:24:19.748414 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 856/1692] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 15 12:24:19 2022] PUT /identity/v3/projects/19e6ec5e336d48f1a10ab8d91c95b1a1/users/68e37396900f49488ebc8b2a4459dea2/roles/9c769b5c82884766b2c510b1f0f1364c => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:24:19.752203 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5feb64-c166-446c-8af0-78f90afb01c6 admin admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:24:19.754478 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-fc5feb64-c166-446c-8af0-78f90afb01c6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:24:19.788892 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:24:19.910255 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 837/1693] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 12:24:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:24:19.920792 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5feb64-c166-446c-8af0-78f90afb01c6 admin admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:24:19.941443 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace35cef-7299-4654-a181-34e1a69e6ffb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:24:19.942349 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace35cef-7299-4654-a181-34e1a69e6ffb admin admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:19.942595 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace35cef-7299-4654-a181-34e1a69e6ffb admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:19.942864 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace35cef-7299-4654-a181-34e1a69e6ffb admin admin] PATH_INFO: `/v3/roles` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:19.943515 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace35cef-7299-4654-a181-34e1a69e6ffb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:24:19.945156 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace35cef-7299-4654-a181-34e1a69e6ffb admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:24:19.961721 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 857/1694] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 12:24:19 2022] GET /identity/v3/roles => generated 1420 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 12:24:19.965652 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18ae51d-9cd6-4522-a03b-283fb292f004 None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:24:19.978847 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18ae51d-9cd6-4522-a03b-283fb292f004 None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:24:19.986900 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d18a39-0ffb-48d2-a243-95a6f6a7738e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:24:19.987736 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d18a39-0ffb-48d2-a243-95a6f6a7738e admin admin] REQUEST_METHOD: `PUT` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:19.988695 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d18a39-0ffb-48d2-a243-95a6f6a7738e admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:19.989120 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d18a39-0ffb-48d2-a243-95a6f6a7738e admin admin] PATH_INFO: `/v3/projects/19e6ec5e336d48f1a10ab8d91c95b1a1/users/68e37396900f49488ebc8b2a4459dea2/roles/9c769b5c82884766b2c510b1f0f1364c` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:19.991258 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d18a39-0ffb-48d2-a243-95a6f6a7738e admin admin] RBAC: Authorizing `identity:create_grant(project_id=19e6ec5e336d48f1a10ab8d91c95b1a1, user_id=68e37396900f49488ebc8b2a4459dea2, role_id=9c769b5c82884766b2c510b1f0f1364c)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:24:19.995061 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d18a39-0ffb-48d2-a243-95a6f6a7738e admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:24:20.012187 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 838/1695] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 15 12:24:19 2022] PUT /identity/v3/projects/19e6ec5e336d48f1a10ab8d91c95b1a1/users/68e37396900f49488ebc8b2a4459dea2/roles/9c769b5c82884766b2c510b1f0f1364c => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:24:20.015950 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace35cef-7299-4654-a181-34e1a69e6ffb admin admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:24:20.017932 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-ace35cef-7299-4654-a181-34e1a69e6ffb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:24:20.029486 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c55ba9-b532-4d29-bac1-7c0dd16a081b None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:20.030981 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c55ba9-b532-4d29-bac1-7c0dd16a081b None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:20.031398 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c55ba9-b532-4d29-bac1-7c0dd16a081b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:20.139134 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-a5c55ba9-b532-4d29-bac1-7c0dd16a081b None None] MFA Rules not processed for user `68e37396900f49488ebc8b2a4459dea2`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:24:20.140949 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-a5c55ba9-b532-4d29-bac1-7c0dd16a081b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:24:20.212232 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace35cef-7299-4654-a181-34e1a69e6ffb admin admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:24:20.341597 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30fa8345-952b-439f-8332-d9808704be89 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfbb4d6809064f9cb9e06626ed7d6814', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '540a32ef88764e8c8d9201ae22f6ddaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:24:20.342272 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fa8345-952b-439f-8332-d9808704be89 service neutron] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:20.342556 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fa8345-952b-439f-8332-d9808704be89 service neutron] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:20.342825 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30fa8345-952b-439f-8332-d9808704be89 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:20.355868 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 839/1696] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 12:24:20 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 333 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:24:20.358815 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c55ba9-b532-4d29-bac1-7c0dd16a081b None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:24:20.360581 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-a5c55ba9-b532-4d29-bac1-7c0dd16a081b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:24:20.419545 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:24:20.480737 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c55ba9-b532-4d29-bac1-7c0dd16a081b None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:24:20.485937 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa50b3d-400a-44ad-9047-49c429849a07 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8191d3ccb874529b32f8d0026db629d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '540a32ef88764e8c8d9201ae22f6ddaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:24:20.486834 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa50b3d-400a-44ad-9047-49c429849a07 service cinder] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:20.487275 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa50b3d-400a-44ad-9047-49c429849a07 service cinder] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:20.487821 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa50b3d-400a-44ad-9047-49c429849a07 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:20.516019 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30fa8345-952b-439f-8332-d9808704be89 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:24:20.517524 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30fa8345-952b-439f-8332-d9808704be89 service neutron] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:24:20.532891 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa50b3d-400a-44ad-9047-49c429849a07 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:24:20.535898 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa50b3d-400a-44ad-9047-49c429849a07 service cinder] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:24:20.552246 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 858/1697] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 12:24:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 537 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 12:24:20.561767 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 840/1698] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 12:24:20 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 205 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 12:24:20.694912 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30fa8345-952b-439f-8332-d9808704be89 service neutron] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:24:20.705686 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-30fa8345-952b-439f-8332-d9808704be89 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:24:20.738450 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30fa8345-952b-439f-8332-d9808704be89 service neutron] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:24:20.768448 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa50b3d-400a-44ad-9047-49c429849a07 service cinder] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:24:20.770069 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-faa50b3d-400a-44ad-9047-49c429849a07 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:24:20.776247 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d245b2-fede-4ca9-895a-aa0f75b8c460 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:24:20.781401 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d245b2-fede-4ca9-895a-aa0f75b8c460 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:20.783212 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d245b2-fede-4ca9-895a-aa0f75b8c460 admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:20.783499 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d245b2-fede-4ca9-895a-aa0f75b8c460 admin admin] PATH_INFO: `/v3/projects/f832edf73ffb484586c4c4db9544cc26` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:20.787904 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d245b2-fede-4ca9-895a-aa0f75b8c460 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f832edf73ffb484586c4c4db9544cc26)` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:24:20.794302 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d245b2-fede-4ca9-895a-aa0f75b8c460 admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:24:20.803989 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa50b3d-400a-44ad-9047-49c429849a07 service cinder] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:24:20.819781 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55de5804-4930-4ef4-9e3e-38cc679f7bd0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8191d3ccb874529b32f8d0026db629d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '540a32ef88764e8c8d9201ae22f6ddaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:24:20.821472 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55de5804-4930-4ef4-9e3e-38cc679f7bd0 service cinder] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:20.823529 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55de5804-4930-4ef4-9e3e-38cc679f7bd0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:20.824106 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55de5804-4930-4ef4-9e3e-38cc679f7bd0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:20.828710 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-55de5804-4930-4ef4-9e3e-38cc679f7bd0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:24:20.897417 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:24:20.998677 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55de5804-4930-4ef4-9e3e-38cc679f7bd0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:24:21.000613 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55de5804-4930-4ef4-9e3e-38cc679f7bd0 service cinder] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:24:21.048188 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 841/1699] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 12:24:20 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 282 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 12:24:21.060708 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-53d245b2-fede-4ca9-895a-aa0f75b8c460 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f832edf73ffb484586c4c4db9544cc26', 'request_id': 'req-53d245b2-fede-4ca9-895a-aa0f75b8c460'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:24:21.062717 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829d8fbf-0228-4687-a2af-270aa02b0367 None None] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:21.063362 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829d8fbf-0228-4687-a2af-270aa02b0367 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:21.064353 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829d8fbf-0228-4687-a2af-270aa02b0367 None None] PATH_INFO: `/` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:21.074883 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 842/1700] 10.222.0.15 () {54 vars in 912 bytes} [Thu Sep 15 12:24:21 2022] GET /identity => generated 270 bytes in 26 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:24:21.081054 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e49dc9-0206-4519-8b37-9603a217330c None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:21.081865 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e49dc9-0206-4519-8b37-9603a217330c None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:21.082908 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e49dc9-0206-4519-8b37-9603a217330c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:21.091862 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-53d245b2-fede-4ca9-895a-aa0f75b8c460 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f832edf73ffb484586c4c4db9544cc26', 'request_id': 'req-53d245b2-fede-4ca9-895a-aa0f75b8c460'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:24:21.105750 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-53d245b2-fede-4ca9-895a-aa0f75b8c460 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f832edf73ffb484586c4c4db9544cc26', 'request_id': 'req-53d245b2-fede-4ca9-895a-aa0f75b8c460'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:24:21.195923 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-53d245b2-fede-4ca9-895a-aa0f75b8c460 admin admin] The token cache is being invalidate because project f832edf73ffb484586c4c4db9544cc26 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81449) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 12:24:21.196331 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-53d245b2-fede-4ca9-895a-aa0f75b8c460 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:24:21.201387 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-53d245b2-fede-4ca9-895a-aa0f75b8c460 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:24:21.208645 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 859/1701] 10.222.0.15 () {58 vars in 1233 bytes} [Thu Sep 15 12:24:20 2022] DELETE /identity/v3/projects/f832edf73ffb484586c4c4db9544cc26 => generated 0 bytes in 514 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:24:21.216246 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d245b2-fede-4ca9-895a-aa0f75b8c460 admin admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:24:21.221244 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-53d245b2-fede-4ca9-895a-aa0f75b8c460 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:24:21.314920 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-96e49dc9-0206-4519-8b37-9603a217330c None None] MFA Rules not processed for user `bfbb4d6809064f9cb9e06626ed7d6814`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:24:21.325663 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-96e49dc9-0206-4519-8b37-9603a217330c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:24:21.588955 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d245b2-fede-4ca9-895a-aa0f75b8c460 admin admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:24:21.603450 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 843/1702] 10.222.0.15 () {58 vars in 1009 bytes} [Thu Sep 15 12:24:21 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 527 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:24:21.611571 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4640abf-34b5-4422-b9fc-d6a0e3abb234 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:24:21.613765 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4640abf-34b5-4422-b9fc-d6a0e3abb234 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:21.614042 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4640abf-34b5-4422-b9fc-d6a0e3abb234 admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:21.614316 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4640abf-34b5-4422-b9fc-d6a0e3abb234 admin admin] PATH_INFO: `/v3/users/41db01ef23f647308a2fbe90526f36f9` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:21.617874 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4640abf-34b5-4422-b9fc-d6a0e3abb234 admin admin] RBAC: Authorizing `identity:delete_user(user_id=41db01ef23f647308a2fbe90526f36f9)` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:24:21.622497 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4640abf-34b5-4422-b9fc-d6a0e3abb234 admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:24:21.755954 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e49dc9-0206-4519-8b37-9603a217330c None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:24:21.757997 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-96e49dc9-0206-4519-8b37-9603a217330c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:24:21.783226 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e49dc9-0206-4519-8b37-9603a217330c None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:24:21.785277 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-b4640abf-34b5-4422-b9fc-d6a0e3abb234 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '41db01ef23f647308a2fbe90526f36f9'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:24:21.795031 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:24:21.796303 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d743e7-cb87-4151-a000-37d2f48ef413 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfbb4d6809064f9cb9e06626ed7d6814', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '540a32ef88764e8c8d9201ae22f6ddaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:24:21.799726 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d743e7-cb87-4151-a000-37d2f48ef413 service neutron] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:21.801195 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d743e7-cb87-4151-a000-37d2f48ef413 service neutron] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:21.802062 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d743e7-cb87-4151-a000-37d2f48ef413 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:21.811709 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-b4640abf-34b5-4422-b9fc-d6a0e3abb234 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '41db01ef23f647308a2fbe90526f36f9'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:24:21.836693 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d743e7-cb87-4151-a000-37d2f48ef413 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:24:21.842823 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d743e7-cb87-4151-a000-37d2f48ef413 service neutron] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:24:21.856530 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-b4640abf-34b5-4422-b9fc-d6a0e3abb234 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '41db01ef23f647308a2fbe90526f36f9'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:24:21.861980 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-b4640abf-34b5-4422-b9fc-d6a0e3abb234 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '41db01ef23f647308a2fbe90526f36f9'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:24:21.880006 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 844/1703] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 12:24:21 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 15 12:24:21.886339 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-b4640abf-34b5-4422-b9fc-d6a0e3abb234 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '41db01ef23f647308a2fbe90526f36f9'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:24:21.901431 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 860/1704] 10.222.0.15 () {58 vars in 1224 bytes} [Thu Sep 15 12:24:21 2022] DELETE /identity/v3/users/41db01ef23f647308a2fbe90526f36f9 => generated 0 bytes in 688 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:24:22.391848 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d743e7-cb87-4151-a000-37d2f48ef413 service neutron] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:24:22.393578 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-00d743e7-cb87-4151-a000-37d2f48ef413 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:24:22.428483 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d743e7-cb87-4151-a000-37d2f48ef413 service neutron] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:24:22.617454 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb67a19a-ad94-48e0-8a2f-c5f65b6142b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:24:22.621999 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb67a19a-ad94-48e0-8a2f-c5f65b6142b3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:22.622496 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb67a19a-ad94-48e0-8a2f-c5f65b6142b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:22.622791 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb67a19a-ad94-48e0-8a2f-c5f65b6142b3 admin admin] PATH_INFO: `/v3/projects/59ace7b38143442bb6a511c14cb13a9a` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:22.624256 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb67a19a-ad94-48e0-8a2f-c5f65b6142b3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=59ace7b38143442bb6a511c14cb13a9a)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:24:22.660858 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb67a19a-ad94-48e0-8a2f-c5f65b6142b3 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:24:22.789417 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-eb67a19a-ad94-48e0-8a2f-c5f65b6142b3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '59ace7b38143442bb6a511c14cb13a9a', 'request_id': 'req-eb67a19a-ad94-48e0-8a2f-c5f65b6142b3'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:24:22.809127 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-eb67a19a-ad94-48e0-8a2f-c5f65b6142b3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '59ace7b38143442bb6a511c14cb13a9a', 'request_id': 'req-eb67a19a-ad94-48e0-8a2f-c5f65b6142b3'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:24:22.814099 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:24:22.823179 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-eb67a19a-ad94-48e0-8a2f-c5f65b6142b3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '59ace7b38143442bb6a511c14cb13a9a', 'request_id': 'req-eb67a19a-ad94-48e0-8a2f-c5f65b6142b3'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:24:22.848385 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-eb67a19a-ad94-48e0-8a2f-c5f65b6142b3 admin admin] The token cache is being invalidate because project 59ace7b38143442bb6a511c14cb13a9a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81448) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 12:24:22.848687 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-eb67a19a-ad94-48e0-8a2f-c5f65b6142b3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:24:22.849599 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-eb67a19a-ad94-48e0-8a2f-c5f65b6142b3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:24:22.854245 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 845/1705] 10.222.0.15 () {58 vars in 1233 bytes} [Thu Sep 15 12:24:22 2022] DELETE /identity/v3/projects/59ace7b38143442bb6a511c14cb13a9a => generated 0 bytes in 463 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:24:22.858976 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539f6857-9c0d-4067-9ee0-42e990157f83 None None] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:22.859455 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539f6857-9c0d-4067-9ee0-42e990157f83 None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:22.859550 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539f6857-9c0d-4067-9ee0-42e990157f83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:22.930137 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.auth.core [None req-539f6857-9c0d-4067-9ee0-42e990157f83 None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81449) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:24:22.941720 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-539f6857-9c0d-4067-9ee0-42e990157f83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:24:23.022015 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9eead45-ee55-478e-a5c8-dd5399a817af None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:23.022364 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9eead45-ee55-478e-a5c8-dd5399a817af None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:23.022643 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9eead45-ee55-478e-a5c8-dd5399a817af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:23.095591 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-d9eead45-ee55-478e-a5c8-dd5399a817af None None] MFA Rules not processed for user `89cc4de76c6a4305af9a9506840e13f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:24:23.097131 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-d9eead45-ee55-478e-a5c8-dd5399a817af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:24:23.146018 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 861/1706] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 12:24:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 288 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:24:23.153253 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034d7ce2-02b7-42ab-87af-b2cae9acb9e5 None None] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:23.153582 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034d7ce2-02b7-42ab-87af-b2cae9acb9e5 None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:23.153880 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034d7ce2-02b7-42ab-87af-b2cae9acb9e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:23.222536 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.auth.core [None req-034d7ce2-02b7-42ab-87af-b2cae9acb9e5 None None] MFA Rules not processed for user `1c42d5435a514bc0880b69fbbc163d32`. Rule list: `[]` (Enabled: `True`). {{(pid=81449) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:24:23.224504 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 846/1707] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 12:24:23 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 12:24:23.226430 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e45accf-a2b1-4df2-b79d-4f30a13dc539 None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:23.226719 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e45accf-a2b1-4df2-b79d-4f30a13dc539 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:23.229211 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e45accf-a2b1-4df2-b79d-4f30a13dc539 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:23.233700 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-034d7ce2-02b7-42ab-87af-b2cae9acb9e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:24:23.345515 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-6e45accf-a2b1-4df2-b79d-4f30a13dc539 None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:24:23.347232 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-6e45accf-a2b1-4df2-b79d-4f30a13dc539 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:24:23.402153 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 847/1708] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 12:24:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:24:23.403134 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e45accf-a2b1-4df2-b79d-4f30a13dc539 None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:24:23.410195 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-6e45accf-a2b1-4df2-b79d-4f30a13dc539 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:24:23.440256 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 862/1709] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 12:24:23 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 293 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:24:23.441631 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034d7ce2-02b7-42ab-87af-b2cae9acb9e5 None None] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:24:23.466097 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034d7ce2-02b7-42ab-87af-b2cae9acb9e5 None None] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:24:23.478340 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ddf6ed1-495a-4429-9dba-0c1883646b2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:24:23.483835 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddf6ed1-495a-4429-9dba-0c1883646b2b admin admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:23.484985 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddf6ed1-495a-4429-9dba-0c1883646b2b admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:23.485271 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddf6ed1-495a-4429-9dba-0c1883646b2b admin admin] PATH_INFO: `/v3/domains` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:23.488239 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ddf6ed1-495a-4429-9dba-0c1883646b2b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81449) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:24:23.488641 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ddf6ed1-495a-4429-9dba-0c1883646b2b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:24:23.491878 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ddf6ed1-495a-4429-9dba-0c1883646b2b admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:24:23.528328 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 863/1710] 10.222.0.15 () {58 vars in 1153 bytes} [Thu Sep 15 12:24:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:24:23.530723 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ddf6ed1-495a-4429-9dba-0c1883646b2b admin admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:24:23.547325 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ddf6ed1-495a-4429-9dba-0c1883646b2b admin admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:24:23.553150 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf79379-5d5e-414f-9a37-585bba4cf137 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:24:23.553861 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf79379-5d5e-414f-9a37-585bba4cf137 admin admin] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:23.554142 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf79379-5d5e-414f-9a37-585bba4cf137 admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:23.554466 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf79379-5d5e-414f-9a37-585bba4cf137 admin admin] PATH_INFO: `/v3/projects` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:23.555450 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf79379-5d5e-414f-9a37-585bba4cf137 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:24:23.563414 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf79379-5d5e-414f-9a37-585bba4cf137 admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:24:23.665031 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 864/1711] 10.222.0.15 () {60 vars in 1153 bytes} [Thu Sep 15 12:24:23 2022] POST /identity/v3/projects => generated 373 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 12:24:23.669126 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf79379-5d5e-414f-9a37-585bba4cf137 admin admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:24:23.675038 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-ecf79379-5d5e-414f-9a37-585bba4cf137 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:24:23.730585 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e45accf-a2b1-4df2-b79d-4f30a13dc539 None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:24:23.772176 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=97,keystone:DELETE=2,keystone:INSERT=1 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:24:23.901137 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75d1157-fd88-4225-96bb-318d3871775f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eface89e261840a2abb4206be26455c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '540a32ef88764e8c8d9201ae22f6ddaf', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:24:23.901953 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75d1157-fd88-4225-96bb-318d3871775f service nova] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:23.902363 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75d1157-fd88-4225-96bb-318d3871775f service nova] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:23.902548 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75d1157-fd88-4225-96bb-318d3871775f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:23.907936 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:24:23.918278 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf79379-5d5e-414f-9a37-585bba4cf137 admin admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:24:23.929070 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a698f2-f2ed-4533-9419-f9763e4f17ce service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f64a058bd5440cb5268f228aa6e392', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '540a32ef88764e8c8d9201ae22f6ddaf', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:24:23.929704 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a698f2-f2ed-4533-9419-f9763e4f17ce service placement] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:23.929986 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a698f2-f2ed-4533-9419-f9763e4f17ce service placement] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:23.930245 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a698f2-f2ed-4533-9419-f9763e4f17ce service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:23.940663 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-b5a698f2-f2ed-4533-9419-f9763e4f17ce service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:24:23.970528 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a698f2-f2ed-4533-9419-f9763e4f17ce service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:24:23.972426 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a698f2-f2ed-4533-9419-f9763e4f17ce service placement] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:24:23.998012 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 865/1712] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 12:24:23 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 12:24:24.000717 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a698f2-f2ed-4533-9419-f9763e4f17ce service placement] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:24:24.038564 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75d1157-fd88-4225-96bb-318d3871775f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:24:24.042901 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75d1157-fd88-4225-96bb-318d3871775f service nova] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:24:24.075708 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 848/1713] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 12:24:23 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 673 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 12:24:24.076743 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75d1157-fd88-4225-96bb-318d3871775f service nova] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:24:24.085243 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-f75d1157-fd88-4225-96bb-318d3871775f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:24:24.125233 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75d1157-fd88-4225-96bb-318d3871775f service nova] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:24:24.135684 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a698f2-f2ed-4533-9419-f9763e4f17ce service placement] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:24:24.141901 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7987b16d-963e-4b1e-a9f3-93e479e10b76 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eface89e261840a2abb4206be26455c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '540a32ef88764e8c8d9201ae22f6ddaf', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:24:24.143680 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7987b16d-963e-4b1e-a9f3-93e479e10b76 service nova] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:24.144650 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b7223f-f5d9-4062-ad13-cb435ed2e01b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:24:24.144832 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7987b16d-963e-4b1e-a9f3-93e479e10b76 service nova] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:24.145038 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7987b16d-963e-4b1e-a9f3-93e479e10b76 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:24.145344 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b7223f-f5d9-4062-ad13-cb435ed2e01b admin admin] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:24.145636 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b7223f-f5d9-4062-ad13-cb435ed2e01b admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:24.145898 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b7223f-f5d9-4062-ad13-cb435ed2e01b admin admin] PATH_INFO: `/v3/users` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:24.147124 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b7223f-f5d9-4062-ad13-cb435ed2e01b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:24:24.148819 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b7223f-f5d9-4062-ad13-cb435ed2e01b admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:24:24.189908 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 866/1714] 10.222.0.15 () {60 vars in 1144 bytes} [Thu Sep 15 12:24:23 2022] POST /identity/v3/users => generated 287 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 12:24:24.195078 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b7223f-f5d9-4062-ad13-cb435ed2e01b admin admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:24:24.200337 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-63b7223f-f5d9-4062-ad13-cb435ed2e01b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:24:24.232258 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b7223f-f5d9-4062-ad13-cb435ed2e01b admin admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:24:24.263868 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fff5872-353a-4ea0-a22b-acb291c7a719 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:24:24.265788 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fff5872-353a-4ea0-a22b-acb291c7a719 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:24.265788 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fff5872-353a-4ea0-a22b-acb291c7a719 admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:24.265788 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fff5872-353a-4ea0-a22b-acb291c7a719 admin admin] PATH_INFO: `/v3/projects/e009b017ce8c40a098d047fdc98b06cd` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:24.266846 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fff5872-353a-4ea0-a22b-acb291c7a719 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e009b017ce8c40a098d047fdc98b06cd)` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:24:24.269136 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fff5872-353a-4ea0-a22b-acb291c7a719 admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:24:24.380835 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7987b16d-963e-4b1e-a9f3-93e479e10b76 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:24:24.387817 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7987b16d-963e-4b1e-a9f3-93e479e10b76 service nova] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:24:24.399737 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-3fff5872-353a-4ea0-a22b-acb291c7a719 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e009b017ce8c40a098d047fdc98b06cd', 'request_id': 'req-3fff5872-353a-4ea0-a22b-acb291c7a719'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:24:24.419002 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-3fff5872-353a-4ea0-a22b-acb291c7a719 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e009b017ce8c40a098d047fdc98b06cd', 'request_id': 'req-3fff5872-353a-4ea0-a22b-acb291c7a719'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:24:24.422186 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 849/1715] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 12:24:24 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 12:24:24.423049 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7987b16d-963e-4b1e-a9f3-93e479e10b76 service nova] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:24:24.435148 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-3fff5872-353a-4ea0-a22b-acb291c7a719 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e009b017ce8c40a098d047fdc98b06cd', 'request_id': 'req-3fff5872-353a-4ea0-a22b-acb291c7a719'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:24:24.445304 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7987b16d-963e-4b1e-a9f3-93e479e10b76 service nova] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:24:24.471498 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-3fff5872-353a-4ea0-a22b-acb291c7a719 admin admin] The token cache is being invalidate because project e009b017ce8c40a098d047fdc98b06cd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81449) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 12:24:24.471713 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-3fff5872-353a-4ea0-a22b-acb291c7a719 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:24:24.473426 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-3fff5872-353a-4ea0-a22b-acb291c7a719 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:24:24.481311 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 867/1716] 10.222.0.15 () {58 vars in 1233 bytes} [Thu Sep 15 12:24:24 2022] DELETE /identity/v3/projects/e009b017ce8c40a098d047fdc98b06cd => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:24:24.486757 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50895db9-721f-41bb-a5b5-768c8451741a None None] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:24.488058 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50895db9-721f-41bb-a5b5-768c8451741a None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:24.488058 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50895db9-721f-41bb-a5b5-768c8451741a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:24.567724 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.auth.core [None req-50895db9-721f-41bb-a5b5-768c8451741a None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81449) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:24:24.569227 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-50895db9-721f-41bb-a5b5-768c8451741a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:24:24.607789 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6fe945-36a2-4015-87a4-fda19ca7eace admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:24:24.609297 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6fe945-36a2-4015-87a4-fda19ca7eace admin admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:24.609576 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6fe945-36a2-4015-87a4-fda19ca7eace admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:24.609843 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6fe945-36a2-4015-87a4-fda19ca7eace admin admin] PATH_INFO: `/v3/roles` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:24.610487 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd6fe945-36a2-4015-87a4-fda19ca7eace admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:24:24.612285 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd6fe945-36a2-4015-87a4-fda19ca7eace admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:24:24.623756 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:24:24.669356 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 850/1717] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 12:24:24 2022] GET /identity/v3/roles => generated 1420 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 12:24:24.673945 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6fe945-36a2-4015-87a4-fda19ca7eace admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:24:24.679374 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-fd6fe945-36a2-4015-87a4-fda19ca7eace admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:24:24.734689 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6fe945-36a2-4015-87a4-fda19ca7eace admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:24:24.757407 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09abaf96-7958-40f0-98bd-f5d7c91bec71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:24:24.758451 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09abaf96-7958-40f0-98bd-f5d7c91bec71 admin admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:24.758971 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09abaf96-7958-40f0-98bd-f5d7c91bec71 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:24.759393 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09abaf96-7958-40f0-98bd-f5d7c91bec71 admin admin] PATH_INFO: `/v3/roles` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:24.760566 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09abaf96-7958-40f0-98bd-f5d7c91bec71 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:24:24.762628 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09abaf96-7958-40f0-98bd-f5d7c91bec71 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:24:24.793941 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 868/1718] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 12:24:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 307 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:24:24.794888 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 851/1719] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 12:24:24 2022] GET /identity/v3/roles => generated 1420 bytes in 122 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 12:24:24.795591 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1182f325-9cc5-4a48-acb4-f08875e53985 None None] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:24.795882 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1182f325-9cc5-4a48-acb4-f08875e53985 None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:24.796152 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1182f325-9cc5-4a48-acb4-f08875e53985 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:24.800328 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09abaf96-7958-40f0-98bd-f5d7c91bec71 admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:24:24.824638 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09abaf96-7958-40f0-98bd-f5d7c91bec71 admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:24:24.865302 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.auth.core [None req-1182f325-9cc5-4a48-acb4-f08875e53985 None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81449) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:24:24.870309 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df128ded-b469-4b4f-a70c-eabe16227925 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:24:24.871036 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df128ded-b469-4b4f-a70c-eabe16227925 admin admin] REQUEST_METHOD: `PUT` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:24.871723 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df128ded-b469-4b4f-a70c-eabe16227925 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:24.871723 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df128ded-b469-4b4f-a70c-eabe16227925 admin admin] PATH_INFO: `/v3/projects/7399ee4d636f40dc83046efefbff94a3/users/bf614c2c47ba44f7acab0a3c54217959/roles/0679d6628a25438d90dc4bdc0401eda2` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:24.885432 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-1182f325-9cc5-4a48-acb4-f08875e53985 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:24:24.909802 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 869/1720] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 12:24:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:24:24.910661 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df128ded-b469-4b4f-a70c-eabe16227925 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7399ee4d636f40dc83046efefbff94a3, user_id=bf614c2c47ba44f7acab0a3c54217959, role_id=0679d6628a25438d90dc4bdc0401eda2)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:24:24.913645 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df128ded-b469-4b4f-a70c-eabe16227925 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:24:24.917755 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1182f325-9cc5-4a48-acb4-f08875e53985 None None] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:24:24.935276 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1182f325-9cc5-4a48-acb4-f08875e53985 None None] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:24:24.950491 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 852/1721] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 15 12:24:24 2022] PUT /identity/v3/projects/7399ee4d636f40dc83046efefbff94a3/users/bf614c2c47ba44f7acab0a3c54217959/roles/0679d6628a25438d90dc4bdc0401eda2 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:24:24.952452 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cbc974-524b-4203-b07a-0ab3d81aa871 None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:24.953555 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cbc974-524b-4203-b07a-0ab3d81aa871 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:24.953854 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cbc974-524b-4203-b07a-0ab3d81aa871 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:24.968876 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:24:25.059253 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0320b052-c174-4014-b3f5-924dce13b5d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:24:25.060324 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0320b052-c174-4014-b3f5-924dce13b5d7 admin admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:25.060688 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0320b052-c174-4014-b3f5-924dce13b5d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:25.061249 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0320b052-c174-4014-b3f5-924dce13b5d7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:25.062463 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0320b052-c174-4014-b3f5-924dce13b5d7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81449) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:24:25.062463 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0320b052-c174-4014-b3f5-924dce13b5d7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:24:25.064071 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0320b052-c174-4014-b3f5-924dce13b5d7 admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:24:25.078583 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-b6cbc974-524b-4203-b07a-0ab3d81aa871 None None] MFA Rules not processed for user `bf614c2c47ba44f7acab0a3c54217959`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:24:25.080145 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-b6cbc974-524b-4203-b07a-0ab3d81aa871 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:24:25.086790 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 870/1722] 10.222.0.15 () {58 vars in 1153 bytes} [Thu Sep 15 12:24:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:24:25.092253 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0320b052-c174-4014-b3f5-924dce13b5d7 admin admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:24:25.120340 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0320b052-c174-4014-b3f5-924dce13b5d7 admin admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:24:25.135221 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe651bf7-42b6-46b5-8f9f-978685a4a63b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:24:25.136263 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe651bf7-42b6-46b5-8f9f-978685a4a63b admin admin] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:25.136561 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe651bf7-42b6-46b5-8f9f-978685a4a63b admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:25.137000 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe651bf7-42b6-46b5-8f9f-978685a4a63b admin admin] PATH_INFO: `/v3/projects` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:25.137752 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe651bf7-42b6-46b5-8f9f-978685a4a63b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:24:25.147344 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe651bf7-42b6-46b5-8f9f-978685a4a63b admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:24:25.220788 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 871/1723] 10.222.0.15 () {60 vars in 1153 bytes} [Thu Sep 15 12:24:25 2022] POST /identity/v3/projects => generated 383 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 12:24:25.222524 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe651bf7-42b6-46b5-8f9f-978685a4a63b admin admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:24:25.298483 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 853/1724] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 12:24:24 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 347 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:24:25.305090 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6cbc974-524b-4203-b07a-0ab3d81aa871 None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:24:25.306982 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-b6cbc974-524b-4203-b07a-0ab3d81aa871 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:24:25.411585 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe651bf7-42b6-46b5-8f9f-978685a4a63b admin admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:24:25.429632 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa469509-c887-4c4d-af1b-902fe86106f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:24:25.430153 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa469509-c887-4c4d-af1b-902fe86106f5 admin admin] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:25.430442 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa469509-c887-4c4d-af1b-902fe86106f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:25.430720 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa469509-c887-4c4d-af1b-902fe86106f5 admin admin] PATH_INFO: `/v3/users` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:25.431364 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa469509-c887-4c4d-af1b-902fe86106f5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:24:25.442990 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa469509-c887-4c4d-af1b-902fe86106f5 admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:24:25.501629 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6cbc974-524b-4203-b07a-0ab3d81aa871 None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:24:25.516896 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 872/1725] 10.222.0.15 () {60 vars in 1144 bytes} [Thu Sep 15 12:24:25 2022] POST /identity/v3/users => generated 292 bytes in 295 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 12:24:25.521024 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa469509-c887-4c4d-af1b-902fe86106f5 admin admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:24:25.556213 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa469509-c887-4c4d-af1b-902fe86106f5 admin admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:24:25.585833 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc94f326-bdce-4919-9d70-f38718a9ed78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:24:25.590619 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc94f326-bdce-4919-9d70-f38718a9ed78 admin admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:25.591808 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc94f326-bdce-4919-9d70-f38718a9ed78 admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:25.592602 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc94f326-bdce-4919-9d70-f38718a9ed78 admin admin] PATH_INFO: `/v3/roles` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:25.596520 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc94f326-bdce-4919-9d70-f38718a9ed78 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:24:25.605802 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc94f326-bdce-4919-9d70-f38718a9ed78 admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:24:25.652169 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5fd2ff-712d-4087-a50d-814b4907c63a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eface89e261840a2abb4206be26455c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '540a32ef88764e8c8d9201ae22f6ddaf', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:24:25.652893 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5fd2ff-712d-4087-a50d-814b4907c63a service nova] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:25.654408 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5fd2ff-712d-4087-a50d-814b4907c63a service nova] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:25.654625 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 873/1726] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 12:24:25 2022] GET /identity/v3/roles => generated 1420 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 12:24:25.654920 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5fd2ff-712d-4087-a50d-814b4907c63a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:25.657322 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc94f326-bdce-4919-9d70-f38718a9ed78 admin admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:24:25.702917 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc94f326-bdce-4919-9d70-f38718a9ed78 admin admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:24:25.736861 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41541754-e87f-4804-8444-80c841e213c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:24:25.740662 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41541754-e87f-4804-8444-80c841e213c3 admin admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:25.742633 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41541754-e87f-4804-8444-80c841e213c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:25.743206 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:24:25.744274 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41541754-e87f-4804-8444-80c841e213c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:25.749584 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41541754-e87f-4804-8444-80c841e213c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:24:25.761954 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41541754-e87f-4804-8444-80c841e213c3 admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:24:25.796174 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:24:25.811592 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5fd2ff-712d-4087-a50d-814b4907c63a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:24:25.816036 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5fd2ff-712d-4087-a50d-814b4907c63a service nova] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:24:25.830035 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 874/1727] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 12:24:25 2022] GET /identity/v3/roles => generated 1420 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 12:24:25.832026 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41541754-e87f-4804-8444-80c841e213c3 admin admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:24:25.837407 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 854/1728] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 12:24:25 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 533 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 15 12:24:25.911803 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41541754-e87f-4804-8444-80c841e213c3 admin admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:24:25.918599 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5fd2ff-712d-4087-a50d-814b4907c63a service nova] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:24:25.919970 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-9e5fd2ff-712d-4087-a50d-814b4907c63a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:24:25.932027 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2519d4-d036-4f30-93c7-ae9f0b38683b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:24:25.932836 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2519d4-d036-4f30-93c7-ae9f0b38683b admin admin] REQUEST_METHOD: `PUT` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:25.933723 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2519d4-d036-4f30-93c7-ae9f0b38683b admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:25.933723 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2519d4-d036-4f30-93c7-ae9f0b38683b admin admin] PATH_INFO: `/v3/projects/ef1b2914f1a64c2e99661187047fbd1f/users/28e3a843ae9741098fd85a5d111e3f19/roles/0679d6628a25438d90dc4bdc0401eda2` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:25.963047 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5fd2ff-712d-4087-a50d-814b4907c63a service nova] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:24:25.985816 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f2519d4-d036-4f30-93c7-ae9f0b38683b admin admin] RBAC: Authorizing `identity:create_grant(project_id=ef1b2914f1a64c2e99661187047fbd1f, user_id=28e3a843ae9741098fd85a5d111e3f19, role_id=0679d6628a25438d90dc4bdc0401eda2)` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:24:25.989363 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3b1f61-adf3-490c-b9b5-45507a808ef9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:24:25.990088 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3b1f61-adf3-490c-b9b5-45507a808ef9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:25.990378 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3b1f61-adf3-490c-b9b5-45507a808ef9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:25.990654 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3b1f61-adf3-490c-b9b5-45507a808ef9 admin admin] PATH_INFO: `/v3/users/d8bbf7fbe7e74d058e4f8541f12c10c1` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:25.992432 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f2519d4-d036-4f30-93c7-ae9f0b38683b admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:24:25.993522 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3b1f61-adf3-490c-b9b5-45507a808ef9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d8bbf7fbe7e74d058e4f8541f12c10c1)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:24:26.001281 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3b1f61-adf3-490c-b9b5-45507a808ef9 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:24:26.031144 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 875/1729] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 15 12:24:25 2022] PUT /identity/v3/projects/ef1b2914f1a64c2e99661187047fbd1f/users/28e3a843ae9741098fd85a5d111e3f19/roles/0679d6628a25438d90dc4bdc0401eda2 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:24:26.033235 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d65233-bd18-4371-b0df-d60f4713f8c6 None None] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:26.034319 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d65233-bd18-4371-b0df-d60f4713f8c6 None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:26.034319 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d65233-bd18-4371-b0df-d60f4713f8c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:26.127303 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.auth.core [None req-62d65233-bd18-4371-b0df-d60f4713f8c6 None None] MFA Rules not processed for user `28e3a843ae9741098fd85a5d111e3f19`. Rule list: `[]` (Enabled: `True`). {{(pid=81449) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:24:26.128771 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-62d65233-bd18-4371-b0df-d60f4713f8c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:24:26.161944 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-ca3b1f61-adf3-490c-b9b5-45507a808ef9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd8bbf7fbe7e74d058e4f8541f12c10c1'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:24:26.209416 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-ca3b1f61-adf3-490c-b9b5-45507a808ef9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd8bbf7fbe7e74d058e4f8541f12c10c1'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:24:26.230054 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-ca3b1f61-adf3-490c-b9b5-45507a808ef9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd8bbf7fbe7e74d058e4f8541f12c10c1'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:24:26.233738 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-ca3b1f61-adf3-490c-b9b5-45507a808ef9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd8bbf7fbe7e74d058e4f8541f12c10c1'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:24:26.255727 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-ca3b1f61-adf3-490c-b9b5-45507a808ef9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd8bbf7fbe7e74d058e4f8541f12c10c1'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:24:26.272448 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 855/1730] 10.222.0.15 () {58 vars in 1224 bytes} [Thu Sep 15 12:24:25 2022] DELETE /identity/v3/users/d8bbf7fbe7e74d058e4f8541f12c10c1 => generated 0 bytes in 354 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:24:26.276900 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02a7fc5-da24-49ef-bee9-6d40b7f75fc8 None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:26.277394 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02a7fc5-da24-49ef-bee9-6d40b7f75fc8 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:26.277853 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02a7fc5-da24-49ef-bee9-6d40b7f75fc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:26.335337 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-a02a7fc5-da24-49ef-bee9-6d40b7f75fc8 None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:24:26.336604 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-a02a7fc5-da24-49ef-bee9-6d40b7f75fc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:24:26.357719 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 876/1731] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 12:24:26 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 326 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:24:26.363513 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d65233-bd18-4371-b0df-d60f4713f8c6 None None] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:24:26.365511 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-62d65233-bd18-4371-b0df-d60f4713f8c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:24:26.584407 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 856/1732] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 12:24:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:24:26.588628 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02a7fc5-da24-49ef-bee9-6d40b7f75fc8 None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:24:26.591688 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-a02a7fc5-da24-49ef-bee9-6d40b7f75fc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:24:26.632357 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d65233-bd18-4371-b0df-d60f4713f8c6 None None] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:24:26.641589 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02a7fc5-da24-49ef-bee9-6d40b7f75fc8 None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:24:26.655122 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a5bbc5-4438-40ec-9f5f-f9a44b5fbdde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:24:26.655593 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066f926e-3be5-45a6-907f-01cd2804a02b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:24:26.656223 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066f926e-3be5-45a6-907f-01cd2804a02b admin admin] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:26.656497 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066f926e-3be5-45a6-907f-01cd2804a02b admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:26.656789 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066f926e-3be5-45a6-907f-01cd2804a02b admin admin] PATH_INFO: `/v3/projects` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:26.657438 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066f926e-3be5-45a6-907f-01cd2804a02b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:24:26.659368 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066f926e-3be5-45a6-907f-01cd2804a02b admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:24:26.660232 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a5bbc5-4438-40ec-9f5f-f9a44b5fbdde admin admin] REQUEST_METHOD: `DELETE` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:26.660627 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a5bbc5-4438-40ec-9f5f-f9a44b5fbdde admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:26.661182 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a5bbc5-4438-40ec-9f5f-f9a44b5fbdde admin admin] PATH_INFO: `/v3/users/bf614c2c47ba44f7acab0a3c54217959` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:26.665197 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a5bbc5-4438-40ec-9f5f-f9a44b5fbdde admin admin] RBAC: Authorizing `identity:delete_user(user_id=bf614c2c47ba44f7acab0a3c54217959)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:24:26.668231 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a5bbc5-4438-40ec-9f5f-f9a44b5fbdde admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:24:26.747965 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 877/1733] 10.222.0.15 () {60 vars in 1153 bytes} [Thu Sep 15 12:24:26 2022] POST /identity/v3/projects => generated 381 bytes in 385 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 12:24:26.758469 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066f926e-3be5-45a6-907f-01cd2804a02b admin admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:24:26.760313 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-066f926e-3be5-45a6-907f-01cd2804a02b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:24:26.784405 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:24:26.810558 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-a6a5bbc5-4438-40ec-9f5f-f9a44b5fbdde admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bf614c2c47ba44f7acab0a3c54217959'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:24:26.845764 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-a6a5bbc5-4438-40ec-9f5f-f9a44b5fbdde admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bf614c2c47ba44f7acab0a3c54217959'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:24:26.861223 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-a6a5bbc5-4438-40ec-9f5f-f9a44b5fbdde admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bf614c2c47ba44f7acab0a3c54217959'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:24:26.862038 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-a6a5bbc5-4438-40ec-9f5f-f9a44b5fbdde admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bf614c2c47ba44f7acab0a3c54217959'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:24:26.879819 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-a6a5bbc5-4438-40ec-9f5f-f9a44b5fbdde admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bf614c2c47ba44f7acab0a3c54217959'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:24:26.890169 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 857/1734] 10.222.0.15 () {58 vars in 1224 bytes} [Thu Sep 15 12:24:26 2022] DELETE /identity/v3/users/bf614c2c47ba44f7acab0a3c54217959 => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:24:26.891634 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a5bbc5-4438-40ec-9f5f-f9a44b5fbdde admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:24:26.896395 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-a6a5bbc5-4438-40ec-9f5f-f9a44b5fbdde admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:24:26.919106 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066f926e-3be5-45a6-907f-01cd2804a02b admin admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:24:26.928166 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:24:27.071644 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a5bbc5-4438-40ec-9f5f-f9a44b5fbdde admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:24:27.089169 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8eebd40-1331-48c3-a2f3-957d71b02d10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:24:27.089864 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8eebd40-1331-48c3-a2f3-957d71b02d10 admin admin] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:27.090221 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8eebd40-1331-48c3-a2f3-957d71b02d10 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:27.097562 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8eebd40-1331-48c3-a2f3-957d71b02d10 admin admin] PATH_INFO: `/v3/users` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:27.098566 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8eebd40-1331-48c3-a2f3-957d71b02d10 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:24:27.100869 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8eebd40-1331-48c3-a2f3-957d71b02d10 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:24:27.112882 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2c56ed-1072-43a9-818a-c1857be4a3b0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfbb4d6809064f9cb9e06626ed7d6814', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '540a32ef88764e8c8d9201ae22f6ddaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:24:27.113522 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2c56ed-1072-43a9-818a-c1857be4a3b0 service neutron] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:27.113995 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2c56ed-1072-43a9-818a-c1857be4a3b0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:27.114420 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2c56ed-1072-43a9-818a-c1857be4a3b0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:27.121706 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-3b2c56ed-1072-43a9-818a-c1857be4a3b0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:24:27.147523 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 858/1735] 10.222.0.15 () {60 vars in 1144 bytes} [Thu Sep 15 12:24:26 2022] POST /identity/v3/users => generated 297 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 12:24:27.149013 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec72495-a4bf-4d5d-9c7f-efe00cba718a None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:27.154777 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec72495-a4bf-4d5d-9c7f-efe00cba718a None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:27.155323 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec72495-a4bf-4d5d-9c7f-efe00cba718a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:27.159044 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b2c56ed-1072-43a9-818a-c1857be4a3b0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:24:27.161696 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b2c56ed-1072-43a9-818a-c1857be4a3b0 service neutron] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:24:27.198531 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 878/1736] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 12:24:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 450 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 12:24:27.201173 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73f9ee2-3641-4d92-bbad-b50ce3861d85 None None] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:27.201511 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73f9ee2-3641-4d92-bbad-b50ce3861d85 None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:27.201892 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73f9ee2-3641-4d92-bbad-b50ce3861d85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:27.231598 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-6ec72495-a4bf-4d5d-9c7f-efe00cba718a None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:24:27.231598 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-6ec72495-a4bf-4d5d-9c7f-efe00cba718a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:24:27.254987 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 859/1737] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 12:24:27 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:24:27.257098 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec72495-a4bf-4d5d-9c7f-efe00cba718a None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:24:27.279801 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.auth.core [None req-a73f9ee2-3641-4d92-bbad-b50ce3861d85 None None] MFA Rules not processed for user `4ead4be9666942f2b81cbb0f9ab69325`. Rule list: `[]` (Enabled: `True`). {{(pid=81449) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:24:27.282130 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-a73f9ee2-3641-4d92-bbad-b50ce3861d85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:24:27.290724 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec72495-a4bf-4d5d-9c7f-efe00cba718a None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:24:27.306505 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a51e13-1d1e-4225-a4ac-27a680aec4b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:24:27.309200 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a51e13-1d1e-4225-a4ac-27a680aec4b4 admin admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:27.309519 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a51e13-1d1e-4225-a4ac-27a680aec4b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:27.309790 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a51e13-1d1e-4225-a4ac-27a680aec4b4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:27.310986 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a51e13-1d1e-4225-a4ac-27a680aec4b4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:24:27.318552 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a51e13-1d1e-4225-a4ac-27a680aec4b4 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:24:27.352633 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 860/1738] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 12:24:27 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 12:24:27.353706 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a51e13-1d1e-4225-a4ac-27a680aec4b4 admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:24:27.357594 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-c7a51e13-1d1e-4225-a4ac-27a680aec4b4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:24:27.391305 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a51e13-1d1e-4225-a4ac-27a680aec4b4 admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:24:27.401710 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a01710-d757-4ffb-bc74-88a1fc59b9a5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfbb4d6809064f9cb9e06626ed7d6814', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '540a32ef88764e8c8d9201ae22f6ddaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:24:27.402286 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a01710-d757-4ffb-bc74-88a1fc59b9a5 service neutron] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:27.403048 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a01710-d757-4ffb-bc74-88a1fc59b9a5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:27.403166 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a01710-d757-4ffb-bc74-88a1fc59b9a5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:27.428012 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a01710-d757-4ffb-bc74-88a1fc59b9a5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:24:27.429837 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a01710-d757-4ffb-bc74-88a1fc59b9a5 service neutron] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:24:27.451680 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 879/1739] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 12:24:27 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 245 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:24:27.451680 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73f9ee2-3641-4d92-bbad-b50ce3861d85 None None] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:24:27.477066 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 861/1740] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 12:24:27 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 12:24:27.478354 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a01710-d757-4ffb-bc74-88a1fc59b9a5 service neutron] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:24:27.486104 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-70a01710-d757-4ffb-bc74-88a1fc59b9a5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:24:27.489719 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73f9ee2-3641-4d92-bbad-b50ce3861d85 None None] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:24:27.500042 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1536ac18-4202-49a5-8a7e-784f2f8ae9ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:24:27.500937 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1536ac18-4202-49a5-8a7e-784f2f8ae9ba admin admin] REQUEST_METHOD: `PUT` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:27.501369 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1536ac18-4202-49a5-8a7e-784f2f8ae9ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:27.501979 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1536ac18-4202-49a5-8a7e-784f2f8ae9ba admin admin] PATH_INFO: `/v3/projects/c461164ac5f64ca49d822a8f5b2e6b12/users/ad4503ea3b454d0a8bd7bb7ee35646e5/roles/9c769b5c82884766b2c510b1f0f1364c` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:27.544347 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1536ac18-4202-49a5-8a7e-784f2f8ae9ba admin admin] RBAC: Authorizing `identity:create_grant(project_id=c461164ac5f64ca49d822a8f5b2e6b12, user_id=ad4503ea3b454d0a8bd7bb7ee35646e5, role_id=9c769b5c82884766b2c510b1f0f1364c)` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:24:27.549093 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1536ac18-4202-49a5-8a7e-784f2f8ae9ba admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:24:27.577606 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 880/1741] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 15 12:24:27 2022] PUT /identity/v3/projects/c461164ac5f64ca49d822a8f5b2e6b12/users/ad4503ea3b454d0a8bd7bb7ee35646e5/roles/9c769b5c82884766b2c510b1f0f1364c => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:24:27.584842 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1536ac18-4202-49a5-8a7e-784f2f8ae9ba admin admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:24:27.720199 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a01710-d757-4ffb-bc74-88a1fc59b9a5 service neutron] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:24:27.756742 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:24:27.786368 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1536ac18-4202-49a5-8a7e-784f2f8ae9ba admin admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:24:27.798573 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d766ae-5ec6-4de9-a9c8-fe83e5090e47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:24:27.799271 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d766ae-5ec6-4de9-a9c8-fe83e5090e47 admin admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:27.799493 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d766ae-5ec6-4de9-a9c8-fe83e5090e47 admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:27.799763 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d766ae-5ec6-4de9-a9c8-fe83e5090e47 admin admin] PATH_INFO: `/v3/roles` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:27.808353 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d766ae-5ec6-4de9-a9c8-fe83e5090e47 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:24:27.815886 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d766ae-5ec6-4de9-a9c8-fe83e5090e47 admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:24:27.863562 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 881/1742] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 12:24:27 2022] GET /identity/v3/roles => generated 1420 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 12:24:27.867450 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d766ae-5ec6-4de9-a9c8-fe83e5090e47 admin admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:24:27.888482 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d766ae-5ec6-4de9-a9c8-fe83e5090e47 admin admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:24:27.905098 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9820829d-af95-4070-bbd3-ec242e157b69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:24:27.905912 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9820829d-af95-4070-bbd3-ec242e157b69 admin admin] REQUEST_METHOD: `PUT` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:27.906070 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9820829d-af95-4070-bbd3-ec242e157b69 admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:27.906344 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9820829d-af95-4070-bbd3-ec242e157b69 admin admin] PATH_INFO: `/v3/projects/c461164ac5f64ca49d822a8f5b2e6b12/users/ad4503ea3b454d0a8bd7bb7ee35646e5/roles/9c769b5c82884766b2c510b1f0f1364c` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:27.908631 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970308fb-733f-480a-91ab-1eaca564804b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eface89e261840a2abb4206be26455c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '540a32ef88764e8c8d9201ae22f6ddaf', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:24:27.909336 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970308fb-733f-480a-91ab-1eaca564804b service nova] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:27.909609 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970308fb-733f-480a-91ab-1eaca564804b service nova] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:27.909922 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970308fb-733f-480a-91ab-1eaca564804b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:27.910585 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9820829d-af95-4070-bbd3-ec242e157b69 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c461164ac5f64ca49d822a8f5b2e6b12, user_id=ad4503ea3b454d0a8bd7bb7ee35646e5, role_id=9c769b5c82884766b2c510b1f0f1364c)` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:24:27.914373 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9820829d-af95-4070-bbd3-ec242e157b69 admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:24:27.933715 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:24:27.940403 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 882/1743] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 15 12:24:27 2022] PUT /identity/v3/projects/c461164ac5f64ca49d822a8f5b2e6b12/users/ad4503ea3b454d0a8bd7bb7ee35646e5/roles/9c769b5c82884766b2c510b1f0f1364c => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:24:27.958233 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b82e57-6017-4e7d-90c0-bcab365c8140 None None] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:27.958233 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b82e57-6017-4e7d-90c0-bcab365c8140 None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:27.958233 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b82e57-6017-4e7d-90c0-bcab365c8140 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:28.015247 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.auth.core [None req-d7b82e57-6017-4e7d-90c0-bcab365c8140 None None] MFA Rules not processed for user `ad4503ea3b454d0a8bd7bb7ee35646e5`. Rule list: `[]` (Enabled: `True`). {{(pid=81449) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:24:28.016942 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-d7b82e57-6017-4e7d-90c0-bcab365c8140 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:24:28.078832 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-970308fb-733f-480a-91ab-1eaca564804b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:24:28.081139 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-970308fb-733f-480a-91ab-1eaca564804b service nova] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:24:28.093049 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 862/1744] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 12:24:27 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 615 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 12:24:28.153064 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 883/1745] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 12:24:27 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 12:24:28.164546 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970308fb-733f-480a-91ab-1eaca564804b service nova] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:24:28.167977 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-970308fb-733f-480a-91ab-1eaca564804b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:24:28.265464 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970308fb-733f-480a-91ab-1eaca564804b service nova] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:24:28.367106 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290d8b95-94e0-4ba4-a107-7218e66f1743 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8191d3ccb874529b32f8d0026db629d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '540a32ef88764e8c8d9201ae22f6ddaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:24:28.368211 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290d8b95-94e0-4ba4-a107-7218e66f1743 service cinder] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:28.368618 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290d8b95-94e0-4ba4-a107-7218e66f1743 service cinder] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:28.369037 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290d8b95-94e0-4ba4-a107-7218e66f1743 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:28.394228 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-290d8b95-94e0-4ba4-a107-7218e66f1743 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:24:28.399986 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-290d8b95-94e0-4ba4-a107-7218e66f1743 service cinder] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:24:28.437333 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 863/1746] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 12:24:28 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 274 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 12:24:28.523916 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b82e57-6017-4e7d-90c0-bcab365c8140 None None] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:24:28.526115 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-d7b82e57-6017-4e7d-90c0-bcab365c8140 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:24:28.614638 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290d8b95-94e0-4ba4-a107-7218e66f1743 service cinder] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:24:28.627221 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290d8b95-94e0-4ba4-a107-7218e66f1743 service cinder] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:24:28.642837 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b671eb9b-ecc8-4421-803c-57345ce13663 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8191d3ccb874529b32f8d0026db629d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '540a32ef88764e8c8d9201ae22f6ddaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:24:28.648437 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b671eb9b-ecc8-4421-803c-57345ce13663 service cinder] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:28.649620 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b671eb9b-ecc8-4421-803c-57345ce13663 service cinder] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:28.653091 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b671eb9b-ecc8-4421-803c-57345ce13663 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:28.664126 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-b671eb9b-ecc8-4421-803c-57345ce13663 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:24:28.677719 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:24:28.711914 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b82e57-6017-4e7d-90c0-bcab365c8140 None None] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:24:28.722110 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9dfa29-a8a1-474e-b707-4908d5f9bdf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:24:28.722962 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9dfa29-a8a1-474e-b707-4908d5f9bdf8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:28.734294 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9dfa29-a8a1-474e-b707-4908d5f9bdf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:28.734294 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9dfa29-a8a1-474e-b707-4908d5f9bdf8 admin admin] PATH_INFO: `/v3/projects/7399ee4d636f40dc83046efefbff94a3` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:28.738309 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9dfa29-a8a1-474e-b707-4908d5f9bdf8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7399ee4d636f40dc83046efefbff94a3)` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:24:28.745718 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9dfa29-a8a1-474e-b707-4908d5f9bdf8 admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:24:28.784596 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b671eb9b-ecc8-4421-803c-57345ce13663 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:24:28.787239 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b671eb9b-ecc8-4421-803c-57345ce13663 service cinder] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:24:28.893166 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-ff9dfa29-a8a1-474e-b707-4908d5f9bdf8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7399ee4d636f40dc83046efefbff94a3', 'request_id': 'req-ff9dfa29-a8a1-474e-b707-4908d5f9bdf8'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:24:28.913346 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 864/1747] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 12:24:28 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 12:24:28.915844 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-ff9dfa29-a8a1-474e-b707-4908d5f9bdf8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7399ee4d636f40dc83046efefbff94a3', 'request_id': 'req-ff9dfa29-a8a1-474e-b707-4908d5f9bdf8'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:24:28.932196 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-ff9dfa29-a8a1-474e-b707-4908d5f9bdf8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7399ee4d636f40dc83046efefbff94a3', 'request_id': 'req-ff9dfa29-a8a1-474e-b707-4908d5f9bdf8'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:24:29.016933 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b671eb9b-ecc8-4421-803c-57345ce13663 service cinder] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:24:29.020420 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-b671eb9b-ecc8-4421-803c-57345ce13663 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:24:29.066287 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b671eb9b-ecc8-4421-803c-57345ce13663 service cinder] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:24:29.068222 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-ff9dfa29-a8a1-474e-b707-4908d5f9bdf8 admin admin] The token cache is being invalidate because project 7399ee4d636f40dc83046efefbff94a3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81449) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 12:24:29.074935 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-ff9dfa29-a8a1-474e-b707-4908d5f9bdf8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:24:29.077899 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-ff9dfa29-a8a1-474e-b707-4908d5f9bdf8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:24:29.087489 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 884/1748] 10.222.0.15 () {58 vars in 1233 bytes} [Thu Sep 15 12:24:28 2022] DELETE /identity/v3/projects/7399ee4d636f40dc83046efefbff94a3 => generated 0 bytes in 564 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:24:29.093367 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619f7e33-f45f-4dff-a91e-f7ec028be507 None None] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:29.094391 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619f7e33-f45f-4dff-a91e-f7ec028be507 None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:29.095305 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619f7e33-f45f-4dff-a91e-f7ec028be507 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:29.164859 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.auth.core [None req-619f7e33-f45f-4dff-a91e-f7ec028be507 None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81449) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:24:29.167054 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-619f7e33-f45f-4dff-a91e-f7ec028be507 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:24:29.186985 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:24:29.270408 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d916b74-496a-4548-ac46-0179319c5462 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:24:29.271149 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d916b74-496a-4548-ac46-0179319c5462 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:29.271430 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d916b74-496a-4548-ac46-0179319c5462 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:29.271702 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d916b74-496a-4548-ac46-0179319c5462 admin admin] PATH_INFO: `/v3/projects/a0d7744eae56407d9fb9a126aa2b712c` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:29.274565 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d916b74-496a-4548-ac46-0179319c5462 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a0d7744eae56407d9fb9a126aa2b712c)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:24:29.276567 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d916b74-496a-4548-ac46-0179319c5462 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:24:29.458198 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-1d916b74-496a-4548-ac46-0179319c5462 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a0d7744eae56407d9fb9a126aa2b712c', 'request_id': 'req-1d916b74-496a-4548-ac46-0179319c5462'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:24:29.465683 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 885/1749] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 12:24:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 374 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:24:29.470376 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0edebb8-260d-430c-aaee-7868cd8047f5 None None] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:29.470684 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0edebb8-260d-430c-aaee-7868cd8047f5 None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:29.471392 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0edebb8-260d-430c-aaee-7868cd8047f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:29.477335 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-1d916b74-496a-4548-ac46-0179319c5462 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a0d7744eae56407d9fb9a126aa2b712c', 'request_id': 'req-1d916b74-496a-4548-ac46-0179319c5462'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:24:29.499107 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-1d916b74-496a-4548-ac46-0179319c5462 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a0d7744eae56407d9fb9a126aa2b712c', 'request_id': 'req-1d916b74-496a-4548-ac46-0179319c5462'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:24:29.531753 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-1d916b74-496a-4548-ac46-0179319c5462 admin admin] The token cache is being invalidate because project a0d7744eae56407d9fb9a126aa2b712c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81448) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 12:24:29.534606 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-1d916b74-496a-4548-ac46-0179319c5462 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:24:29.537378 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-1d916b74-496a-4548-ac46-0179319c5462 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:24:29.541890 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.auth.core [None req-f0edebb8-260d-430c-aaee-7868cd8047f5 None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81449) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:24:29.545623 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 865/1750] 10.222.0.15 () {58 vars in 1233 bytes} [Thu Sep 15 12:24:29 2022] DELETE /identity/v3/projects/a0d7744eae56407d9fb9a126aa2b712c => generated 0 bytes in 525 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:24:29.548378 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c7cdd0-31be-4114-b1a7-4fa21da5e15f None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:29.548662 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c7cdd0-31be-4114-b1a7-4fa21da5e15f None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:29.548957 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c7cdd0-31be-4114-b1a7-4fa21da5e15f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:29.559285 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-f0edebb8-260d-430c-aaee-7868cd8047f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:24:29.595324 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-05c7cdd0-31be-4114-b1a7-4fa21da5e15f None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:24:29.597068 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-05c7cdd0-31be-4114-b1a7-4fa21da5e15f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:24:29.729103 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:24:29.801238 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 866/1751] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 12:24:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 255 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 12:24:29.802476 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c7cdd0-31be-4114-b1a7-4fa21da5e15f None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:24:29.804263 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-05c7cdd0-31be-4114-b1a7-4fa21da5e15f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:24:29.810104 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 886/1752] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 12:24:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 341 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:24:29.812260 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795cc1a4-db29-441f-aa49-edfa1536b1c1 None None] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:29.812707 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795cc1a4-db29-441f-aa49-edfa1536b1c1 None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:29.818156 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795cc1a4-db29-441f-aa49-edfa1536b1c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:29.837147 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c7cdd0-31be-4114-b1a7-4fa21da5e15f None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:24:29.855298 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.auth.core [None req-795cc1a4-db29-441f-aa49-edfa1536b1c1 None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81449) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:24:29.858356 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfce1e97-2330-4dc8-a56c-dca2b5df7780 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:24:29.860851 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-795cc1a4-db29-441f-aa49-edfa1536b1c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:24:29.861009 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfce1e97-2330-4dc8-a56c-dca2b5df7780 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:29.863155 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfce1e97-2330-4dc8-a56c-dca2b5df7780 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:29.863789 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfce1e97-2330-4dc8-a56c-dca2b5df7780 admin admin] PATH_INFO: `/v3/users/28e3a843ae9741098fd85a5d111e3f19` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:29.867995 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfce1e97-2330-4dc8-a56c-dca2b5df7780 admin admin] RBAC: Authorizing `identity:delete_user(user_id=28e3a843ae9741098fd85a5d111e3f19)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:24:29.873445 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfce1e97-2330-4dc8-a56c-dca2b5df7780 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:24:29.881666 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 887/1753] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 12:24:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 71 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:24:29.883123 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795cc1a4-db29-441f-aa49-edfa1536b1c1 None None] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:24:29.898616 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795cc1a4-db29-441f-aa49-edfa1536b1c1 None None] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:24:29.911637 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7ab590-1063-4665-b0a1-358f383b92a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:24:29.912484 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7ab590-1063-4665-b0a1-358f383b92a2 admin admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:29.912998 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7ab590-1063-4665-b0a1-358f383b92a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:29.913966 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7ab590-1063-4665-b0a1-358f383b92a2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:29.914803 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc7ab590-1063-4665-b0a1-358f383b92a2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81449) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:24:29.915442 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc7ab590-1063-4665-b0a1-358f383b92a2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:24:29.921154 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc7ab590-1063-4665-b0a1-358f383b92a2 admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:24:29.967041 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 888/1754] 10.222.0.15 () {58 vars in 1153 bytes} [Thu Sep 15 12:24:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:24:29.969370 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7ab590-1063-4665-b0a1-358f383b92a2 admin admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:24:30.010876 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7ab590-1063-4665-b0a1-358f383b92a2 admin admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:24:30.043358 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4cb1a9d-8739-425b-b3ed-59835856f915 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:24:30.044000 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cb1a9d-8739-425b-b3ed-59835856f915 admin admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:30.044278 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cb1a9d-8739-425b-b3ed-59835856f915 admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:30.044552 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cb1a9d-8739-425b-b3ed-59835856f915 admin admin] PATH_INFO: `/v3/domains` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:30.045306 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4cb1a9d-8739-425b-b3ed-59835856f915 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81449) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:24:30.045608 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4cb1a9d-8739-425b-b3ed-59835856f915 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:24:30.049115 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4cb1a9d-8739-425b-b3ed-59835856f915 admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:24:30.052908 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-dfce1e97-2330-4dc8-a56c-dca2b5df7780 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '28e3a843ae9741098fd85a5d111e3f19'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:24:30.079572 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 889/1755] 10.222.0.15 () {58 vars in 1153 bytes} [Thu Sep 15 12:24:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:24:30.080647 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4cb1a9d-8739-425b-b3ed-59835856f915 admin admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:24:30.093206 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-dfce1e97-2330-4dc8-a56c-dca2b5df7780 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '28e3a843ae9741098fd85a5d111e3f19'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:24:30.112704 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-dfce1e97-2330-4dc8-a56c-dca2b5df7780 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '28e3a843ae9741098fd85a5d111e3f19'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:24:30.113682 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-dfce1e97-2330-4dc8-a56c-dca2b5df7780 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '28e3a843ae9741098fd85a5d111e3f19'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:24:30.132795 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-dfce1e97-2330-4dc8-a56c-dca2b5df7780 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '28e3a843ae9741098fd85a5d111e3f19'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:24:30.144030 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4cb1a9d-8739-425b-b3ed-59835856f915 admin admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:24:30.158335 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 867/1756] 10.222.0.15 () {58 vars in 1224 bytes} [Thu Sep 15 12:24:29 2022] DELETE /identity/v3/users/28e3a843ae9741098fd85a5d111e3f19 => generated 0 bytes in 354 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:24:30.158335 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfce1e97-2330-4dc8-a56c-dca2b5df7780 admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:24:30.162133 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-dfce1e97-2330-4dc8-a56c-dca2b5df7780 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:24:30.167864 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a313cb-cac9-4b6b-be0e-1d0e0422a5ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:24:30.175170 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a313cb-cac9-4b6b-be0e-1d0e0422a5ec admin admin] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:30.175500 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a313cb-cac9-4b6b-be0e-1d0e0422a5ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:30.178557 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a313cb-cac9-4b6b-be0e-1d0e0422a5ec admin admin] PATH_INFO: `/v3/projects` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:30.179325 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a313cb-cac9-4b6b-be0e-1d0e0422a5ec admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:24:30.187556 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a313cb-cac9-4b6b-be0e-1d0e0422a5ec admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:24:30.309943 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 890/1757] 10.222.0.15 () {60 vars in 1153 bytes} [Thu Sep 15 12:24:30 2022] POST /identity/v3/projects => generated 403 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 12:24:30.316782 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380a360b-5ead-4787-9426-621e2af5f2b3 None None] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:30.317129 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380a360b-5ead-4787-9426-621e2af5f2b3 None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:30.317344 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380a360b-5ead-4787-9426-621e2af5f2b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:30.331076 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfce1e97-2330-4dc8-a56c-dca2b5df7780 admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:24:30.377182 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.auth.core [None req-380a360b-5ead-4787-9426-621e2af5f2b3 None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81449) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:24:30.377182 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-380a360b-5ead-4787-9426-621e2af5f2b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:24:30.475787 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:24:30.518013 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d4afba-c180-47f4-a405-166e065392cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:24:30.519183 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d4afba-c180-47f4-a405-166e065392cf admin admin] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:30.519578 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d4afba-c180-47f4-a405-166e065392cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:30.519845 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d4afba-c180-47f4-a405-166e065392cf admin admin] PATH_INFO: `/v3/projects` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:30.520958 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d4afba-c180-47f4-a405-166e065392cf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:24:30.524867 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d4afba-c180-47f4-a405-166e065392cf admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:24:30.578103 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:24:30.598520 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 868/1758] 10.222.0.15 () {60 vars in 1153 bytes} [Thu Sep 15 12:24:30 2022] POST /identity/v3/projects => generated 373 bytes in 442 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 12:24:30.599347 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 891/1759] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 12:24:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 289 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:24:30.599464 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d4afba-c180-47f4-a405-166e065392cf admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:24:30.600606 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931db55c-a82a-4c02-89a1-1340861e3deb None None] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:30.601037 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931db55c-a82a-4c02-89a1-1340861e3deb None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:30.601283 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-f4d4afba-c180-47f4-a405-166e065392cf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:24:30.604438 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931db55c-a82a-4c02-89a1-1340861e3deb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:30.660435 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.auth.core [None req-931db55c-a82a-4c02-89a1-1340861e3deb None None] MFA Rules not processed for user `89cc4de76c6a4305af9a9506840e13f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81449) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:24:30.661698 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-931db55c-a82a-4c02-89a1-1340861e3deb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:24:30.803679 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d4afba-c180-47f4-a405-166e065392cf admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:24:30.813266 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7952384f-76e3-4de0-bf08-c24bdf3e36dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:24:30.814293 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7952384f-76e3-4de0-bf08-c24bdf3e36dd admin admin] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:30.814641 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7952384f-76e3-4de0-bf08-c24bdf3e36dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:30.815009 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7952384f-76e3-4de0-bf08-c24bdf3e36dd admin admin] PATH_INFO: `/v3/users` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:30.815739 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7952384f-76e3-4de0-bf08-c24bdf3e36dd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:24:30.817978 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7952384f-76e3-4de0-bf08-c24bdf3e36dd admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:24:30.831167 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 892/1760] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 12:24:30 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:24:30.832106 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931db55c-a82a-4c02-89a1-1340861e3deb None None] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:24:30.866147 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931db55c-a82a-4c02-89a1-1340861e3deb None None] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:24:30.872619 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 869/1761] 10.222.0.15 () {60 vars in 1144 bytes} [Thu Sep 15 12:24:30 2022] POST /identity/v3/users => generated 302 bytes in 271 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 12:24:30.873988 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7952384f-76e3-4de0-bf08-c24bdf3e36dd admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:24:30.877494 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-7952384f-76e3-4de0-bf08-c24bdf3e36dd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:24:30.886745 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2377d000-7e9b-4d34-af20-246bd21ef9ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:24:30.887386 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2377d000-7e9b-4d34-af20-246bd21ef9ed admin admin] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:30.887664 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2377d000-7e9b-4d34-af20-246bd21ef9ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:30.887996 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2377d000-7e9b-4d34-af20-246bd21ef9ed admin admin] PATH_INFO: `/v3/users` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:30.888638 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2377d000-7e9b-4d34-af20-246bd21ef9ed admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:24:30.890615 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2377d000-7e9b-4d34-af20-246bd21ef9ed admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:24:30.970149 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 893/1762] 10.222.0.15 () {60 vars in 1144 bytes} [Thu Sep 15 12:24:30 2022] POST /identity/v3/users => generated 287 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 12:24:30.971649 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2377d000-7e9b-4d34-af20-246bd21ef9ed admin admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:24:31.006940 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2377d000-7e9b-4d34-af20-246bd21ef9ed admin admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:24:31.016428 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714f3025-1ddf-49b0-a813-c4bb97f936a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:24:31.017107 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714f3025-1ddf-49b0-a813-c4bb97f936a8 admin admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:31.017387 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714f3025-1ddf-49b0-a813-c4bb97f936a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:31.017655 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714f3025-1ddf-49b0-a813-c4bb97f936a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:31.018531 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714f3025-1ddf-49b0-a813-c4bb97f936a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:24:31.021635 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-714f3025-1ddf-49b0-a813-c4bb97f936a8 admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:24:31.045582 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 894/1763] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 12:24:30 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 12:24:31.046835 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714f3025-1ddf-49b0-a813-c4bb97f936a8 admin admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:24:31.061857 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714f3025-1ddf-49b0-a813-c4bb97f936a8 admin admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:24:31.071082 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a6728b-0239-406e-a017-e8465578bcc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:24:31.071980 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a6728b-0239-406e-a017-e8465578bcc3 admin admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:31.072508 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a6728b-0239-406e-a017-e8465578bcc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:31.073063 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a6728b-0239-406e-a017-e8465578bcc3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:31.073964 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a6728b-0239-406e-a017-e8465578bcc3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:24:31.076028 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a6728b-0239-406e-a017-e8465578bcc3 admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:24:31.093078 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 895/1764] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 12:24:31 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 12:24:31.094639 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6ba6fa-ad60-4d65-93bc-844838ff52e5 None None] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:31.095078 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6ba6fa-ad60-4d65-93bc-844838ff52e5 None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:31.095492 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6ba6fa-ad60-4d65-93bc-844838ff52e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:31.097315 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7952384f-76e3-4de0-bf08-c24bdf3e36dd admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:24:31.113771 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97c50af-36d2-4b78-8c77-ced19b41aaa2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfbb4d6809064f9cb9e06626ed7d6814', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '540a32ef88764e8c8d9201ae22f6ddaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:24:31.114626 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97c50af-36d2-4b78-8c77-ced19b41aaa2 service neutron] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:31.114921 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97c50af-36d2-4b78-8c77-ced19b41aaa2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:31.115201 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97c50af-36d2-4b78-8c77-ced19b41aaa2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:31.141430 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e97c50af-36d2-4b78-8c77-ced19b41aaa2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:24:31.144819 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e97c50af-36d2-4b78-8c77-ced19b41aaa2 service neutron] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:24:31.177913 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 870/1765] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 12:24:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 12:24:31.181122 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97c50af-36d2-4b78-8c77-ced19b41aaa2 service neutron] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:24:31.198106 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.auth.core [None req-2d6ba6fa-ad60-4d65-93bc-844838ff52e5 None None] MFA Rules not processed for user `527f4dd6031546c785bc6360dacb60af`. Rule list: `[]` (Enabled: `True`). {{(pid=81449) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:24:31.200684 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-2d6ba6fa-ad60-4d65-93bc-844838ff52e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:24:31.236166 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97c50af-36d2-4b78-8c77-ced19b41aaa2 service neutron] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:24:31.245609 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51054a3a-ddb1-4a12-ae4a-54e4f7f48238 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:24:31.246268 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51054a3a-ddb1-4a12-ae4a-54e4f7f48238 admin admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:31.246704 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51054a3a-ddb1-4a12-ae4a-54e4f7f48238 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:31.247071 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51054a3a-ddb1-4a12-ae4a-54e4f7f48238 admin admin] PATH_INFO: `/v3/roles` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:31.247835 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51054a3a-ddb1-4a12-ae4a-54e4f7f48238 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:24:31.255014 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51054a3a-ddb1-4a12-ae4a-54e4f7f48238 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:24:31.274009 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 871/1766] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 12:24:31 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 12:24:31.275226 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51054a3a-ddb1-4a12-ae4a-54e4f7f48238 admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:24:31.296622 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51054a3a-ddb1-4a12-ae4a-54e4f7f48238 admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:24:31.316397 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a407716a-43b6-4dc9-bf37-91216ff4029e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:24:31.317059 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a407716a-43b6-4dc9-bf37-91216ff4029e admin admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:31.317341 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a407716a-43b6-4dc9-bf37-91216ff4029e admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:31.317614 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a407716a-43b6-4dc9-bf37-91216ff4029e admin admin] PATH_INFO: `/v3/roles` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:31.318262 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a407716a-43b6-4dc9-bf37-91216ff4029e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:24:31.319857 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a407716a-43b6-4dc9-bf37-91216ff4029e admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:24:31.342327 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 872/1767] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 12:24:31 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 12:24:31.343190 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a407716a-43b6-4dc9-bf37-91216ff4029e admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:24:31.367559 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a407716a-43b6-4dc9-bf37-91216ff4029e admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:24:31.380424 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5ebc75-a1ef-4fff-a51e-fb87b22509a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:24:31.381311 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5ebc75-a1ef-4fff-a51e-fb87b22509a6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:31.381670 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5ebc75-a1ef-4fff-a51e-fb87b22509a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:31.382022 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5ebc75-a1ef-4fff-a51e-fb87b22509a6 admin admin] PATH_INFO: `/v3/projects/46235ae7cb404d7f8b3817acdca81a68/users/5c5f1c7f322d4ea8a7f62005f4296d64/roles/0679d6628a25438d90dc4bdc0401eda2` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:31.410362 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 896/1768] 10.222.0.15 () {58 vars in 1009 bytes} [Thu Sep 15 12:24:31 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 316 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:24:31.413157 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6ba6fa-ad60-4d65-93bc-844838ff52e5 None None] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:24:31.451208 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5ebc75-a1ef-4fff-a51e-fb87b22509a6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=46235ae7cb404d7f8b3817acdca81a68, user_id=5c5f1c7f322d4ea8a7f62005f4296d64, role_id=0679d6628a25438d90dc4bdc0401eda2)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:24:31.454567 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b5ebc75-a1ef-4fff-a51e-fb87b22509a6 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:24:31.464834 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6ba6fa-ad60-4d65-93bc-844838ff52e5 None None] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:24:31.472162 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e801cb6-fe02-4d07-8522-f54322d52bf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:24:31.472996 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e801cb6-fe02-4d07-8522-f54322d52bf7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:31.474463 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e801cb6-fe02-4d07-8522-f54322d52bf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:31.474463 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e801cb6-fe02-4d07-8522-f54322d52bf7 admin admin] PATH_INFO: `/v3/projects/c1f0f49f343342b0ab454757d2547f6d/users/754a45f1f5ab4e80a7b9f19b39830bbf/roles/0679d6628a25438d90dc4bdc0401eda2` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:31.482853 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 873/1769] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 15 12:24:31 2022] PUT /identity/v3/projects/46235ae7cb404d7f8b3817acdca81a68/users/5c5f1c7f322d4ea8a7f62005f4296d64/roles/0679d6628a25438d90dc4bdc0401eda2 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:24:31.489641 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063d4b37-78c3-46cc-9ce7-ad22a78fb808 None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:31.490416 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063d4b37-78c3-46cc-9ce7-ad22a78fb808 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:31.490904 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063d4b37-78c3-46cc-9ce7-ad22a78fb808 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:31.506094 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e801cb6-fe02-4d07-8522-f54322d52bf7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c1f0f49f343342b0ab454757d2547f6d, user_id=754a45f1f5ab4e80a7b9f19b39830bbf, role_id=0679d6628a25438d90dc4bdc0401eda2)` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:24:31.512082 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e801cb6-fe02-4d07-8522-f54322d52bf7 admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:24:31.533584 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-063d4b37-78c3-46cc-9ce7-ad22a78fb808 None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:24:31.542984 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-063d4b37-78c3-46cc-9ce7-ad22a78fb808 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:24:31.543632 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 897/1770] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 15 12:24:31 2022] PUT /identity/v3/projects/c1f0f49f343342b0ab454757d2547f6d/users/754a45f1f5ab4e80a7b9f19b39830bbf/roles/0679d6628a25438d90dc4bdc0401eda2 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:24:31.547238 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2ca6f8-8a2a-48fe-a27c-194eb5097b3d None None] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:31.547517 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2ca6f8-8a2a-48fe-a27c-194eb5097b3d None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:31.547788 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2ca6f8-8a2a-48fe-a27c-194eb5097b3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:31.597286 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:24:31.608193 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:24:31.620065 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.auth.core [None req-8c2ca6f8-8a2a-48fe-a27c-194eb5097b3d None None] MFA Rules not processed for user `754a45f1f5ab4e80a7b9f19b39830bbf`. Rule list: `[]` (Enabled: `True`). {{(pid=81449) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:24:31.623435 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-8c2ca6f8-8a2a-48fe-a27c-194eb5097b3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:24:31.705645 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 874/1771] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 12:24:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:24:31.706814 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063d4b37-78c3-46cc-9ce7-ad22a78fb808 None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:24:31.717435 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-063d4b37-78c3-46cc-9ce7-ad22a78fb808 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:24:31.829432 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 898/1772] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 12:24:31 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 283 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:24:31.834150 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2ca6f8-8a2a-48fe-a27c-194eb5097b3d None None] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:24:31.941928 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2ca6f8-8a2a-48fe-a27c-194eb5097b3d None None] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:24:31.946204 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e108c61-eb7e-4cd4-bb33-255902c77e24 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfbb4d6809064f9cb9e06626ed7d6814', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '540a32ef88764e8c8d9201ae22f6ddaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:24:31.947165 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e108c61-eb7e-4cd4-bb33-255902c77e24 service neutron] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:31.947595 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e108c61-eb7e-4cd4-bb33-255902c77e24 service neutron] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:31.947967 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e108c61-eb7e-4cd4-bb33-255902c77e24 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:31.966609 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e108c61-eb7e-4cd4-bb33-255902c77e24 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:24:31.973764 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e108c61-eb7e-4cd4-bb33-255902c77e24 service neutron] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:24:31.986828 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063d4b37-78c3-46cc-9ce7-ad22a78fb808 None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:24:31.996998 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918fda7a-c11e-4a3b-bb55-66b982cba4ec service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa0d3dc5b7540d59fdd59bbf64a04b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '540a32ef88764e8c8d9201ae22f6ddaf', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:24:31.998272 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918fda7a-c11e-4a3b-bb55-66b982cba4ec service swift] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:31.999476 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918fda7a-c11e-4a3b-bb55-66b982cba4ec service swift] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:32.000425 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918fda7a-c11e-4a3b-bb55-66b982cba4ec service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:32.002770 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 899/1773] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 12:24:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 12:24:32.004968 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e108c61-eb7e-4cd4-bb33-255902c77e24 service neutron] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:24:32.023791 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e108c61-eb7e-4cd4-bb33-255902c77e24 service neutron] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:24:32.039769 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ac4a58-73b0-48cd-87c8-1f855316b8f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:24:32.041379 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ac4a58-73b0-48cd-87c8-1f855316b8f0 admin admin] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:32.041765 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ac4a58-73b0-48cd-87c8-1f855316b8f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:32.042170 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ac4a58-73b0-48cd-87c8-1f855316b8f0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:32.043371 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ac4a58-73b0-48cd-87c8-1f855316b8f0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:24:32.045715 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ac4a58-73b0-48cd-87c8-1f855316b8f0 admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:24:32.095638 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 900/1774] 10.222.0.15 () {60 vars in 1153 bytes} [Thu Sep 15 12:24:32 2022] POST /identity/v3/projects => generated 371 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 12:24:32.097080 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ac4a58-73b0-48cd-87c8-1f855316b8f0 admin admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:24:32.143881 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-918fda7a-c11e-4a3b-bb55-66b982cba4ec service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:24:32.146508 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-918fda7a-c11e-4a3b-bb55-66b982cba4ec service swift] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:24:32.160466 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 875/1775] 10.222.0.15 () {60 vars in 1384 bytes} [Thu Sep 15 12:24:31 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 454 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 15 12:24:32.162090 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083f2281-d30b-4634-b714-2fbdb8b677ea None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:32.162660 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083f2281-d30b-4634-b714-2fbdb8b677ea None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:32.163791 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083f2281-d30b-4634-b714-2fbdb8b677ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:32.203313 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-083f2281-d30b-4634-b714-2fbdb8b677ea None None] MFA Rules not processed for user `1c42d5435a514bc0880b69fbbc163d32`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:24:32.207306 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-083f2281-d30b-4634-b714-2fbdb8b677ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:24:32.218607 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ac4a58-73b0-48cd-87c8-1f855316b8f0 admin admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:24:32.231202 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5974e7b-fb3f-43f9-8381-0f41e288cb71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:24:32.233300 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5974e7b-fb3f-43f9-8381-0f41e288cb71 admin admin] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:32.233615 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5974e7b-fb3f-43f9-8381-0f41e288cb71 admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:32.233944 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5974e7b-fb3f-43f9-8381-0f41e288cb71 admin admin] PATH_INFO: `/v3/users` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:32.234611 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5974e7b-fb3f-43f9-8381-0f41e288cb71 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:24:32.240275 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5974e7b-fb3f-43f9-8381-0f41e288cb71 admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:24:32.307817 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 901/1776] 10.222.0.15 () {60 vars in 1144 bytes} [Thu Sep 15 12:24:32 2022] POST /identity/v3/users => generated 292 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 12:24:32.310018 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5974e7b-fb3f-43f9-8381-0f41e288cb71 admin admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:24:32.337250 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5974e7b-fb3f-43f9-8381-0f41e288cb71 admin admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:24:32.368998 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959dd27c-631f-46cd-9326-997f199ec4a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:24:32.369641 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959dd27c-631f-46cd-9326-997f199ec4a7 admin admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:32.369921 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959dd27c-631f-46cd-9326-997f199ec4a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:32.371613 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959dd27c-631f-46cd-9326-997f199ec4a7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:32.374257 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-959dd27c-631f-46cd-9326-997f199ec4a7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:24:32.376537 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-959dd27c-631f-46cd-9326-997f199ec4a7 admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:24:32.428572 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 902/1777] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 12:24:32 2022] GET /identity/v3/roles => generated 1420 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 12:24:32.430158 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959dd27c-631f-46cd-9326-997f199ec4a7 admin admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:24:32.447987 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:24:32.458794 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959dd27c-631f-46cd-9326-997f199ec4a7 admin admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:24:32.478769 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33effc2-a54d-4ac5-9f4e-32e831b66777 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:24:32.479765 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33effc2-a54d-4ac5-9f4e-32e831b66777 admin admin] REQUEST_METHOD: `PUT` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:32.481098 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33effc2-a54d-4ac5-9f4e-32e831b66777 admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:32.483717 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33effc2-a54d-4ac5-9f4e-32e831b66777 admin admin] PATH_INFO: `/v3/projects/3007eb79cc2840e988a378f231c79fe3/users/7e07985e44e84a459dcf526e8740fa82/roles/9c769b5c82884766b2c510b1f0f1364c` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:32.497270 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:24:32.520721 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33effc2-a54d-4ac5-9f4e-32e831b66777 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3007eb79cc2840e988a378f231c79fe3, user_id=7e07985e44e84a459dcf526e8740fa82, role_id=9c769b5c82884766b2c510b1f0f1364c)` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:24:32.524565 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33effc2-a54d-4ac5-9f4e-32e831b66777 admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:24:32.562050 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 903/1778] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 15 12:24:32 2022] PUT /identity/v3/projects/3007eb79cc2840e988a378f231c79fe3/users/7e07985e44e84a459dcf526e8740fa82/roles/9c769b5c82884766b2c510b1f0f1364c => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:24:32.567614 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 876/1779] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 12:24:32 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 406 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:24:32.569719 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33effc2-a54d-4ac5-9f4e-32e831b66777 admin admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:24:32.789569 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33effc2-a54d-4ac5-9f4e-32e831b66777 admin admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:24:32.807627 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890bf8d9-63b8-46f9-9502-3724e8e5d22d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:24:32.808275 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890bf8d9-63b8-46f9-9502-3724e8e5d22d admin admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:32.808560 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890bf8d9-63b8-46f9-9502-3724e8e5d22d admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:32.811297 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890bf8d9-63b8-46f9-9502-3724e8e5d22d admin admin] PATH_INFO: `/v3/roles` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:32.812096 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890bf8d9-63b8-46f9-9502-3724e8e5d22d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:24:32.821268 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890bf8d9-63b8-46f9-9502-3724e8e5d22d admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:24:32.864320 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083f2281-d30b-4634-b714-2fbdb8b677ea None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:24:32.865831 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 904/1780] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 12:24:32 2022] GET /identity/v3/roles => generated 1420 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 12:24:32.899433 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083f2281-d30b-4634-b714-2fbdb8b677ea None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:24:32.918649 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48ed824-3027-4c0f-895c-42ceb3201158 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:24:32.919271 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48ed824-3027-4c0f-895c-42ceb3201158 admin admin] REQUEST_METHOD: `PUT` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:32.921156 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48ed824-3027-4c0f-895c-42ceb3201158 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:32.922842 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d48ed824-3027-4c0f-895c-42ceb3201158 admin admin] PATH_INFO: `/v3/projects/3007eb79cc2840e988a378f231c79fe3/users/7e07985e44e84a459dcf526e8740fa82/roles/9c769b5c82884766b2c510b1f0f1364c` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:32.927364 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48ed824-3027-4c0f-895c-42ceb3201158 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3007eb79cc2840e988a378f231c79fe3, user_id=7e07985e44e84a459dcf526e8740fa82, role_id=9c769b5c82884766b2c510b1f0f1364c)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:24:32.935426 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d48ed824-3027-4c0f-895c-42ceb3201158 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:24:32.973467 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 877/1781] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 15 12:24:32 2022] PUT /identity/v3/projects/3007eb79cc2840e988a378f231c79fe3/users/7e07985e44e84a459dcf526e8740fa82/roles/9c769b5c82884766b2c510b1f0f1364c => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 12:24:32.985192 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453b8829-eee5-4527-892d-8fa39ae27dd0 None None] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:32.985488 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453b8829-eee5-4527-892d-8fa39ae27dd0 None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:32.985764 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453b8829-eee5-4527-892d-8fa39ae27dd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:33.087155 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.auth.core [None req-453b8829-eee5-4527-892d-8fa39ae27dd0 None None] MFA Rules not processed for user `7e07985e44e84a459dcf526e8740fa82`. Rule list: `[]` (Enabled: `True`). {{(pid=81449) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:24:33.091986 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-453b8829-eee5-4527-892d-8fa39ae27dd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:24:33.331189 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 905/1782] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 12:24:32 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 343 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:24:33.341140 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48ed824-3027-4c0f-895c-42ceb3201158 admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:24:33.344083 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-d48ed824-3027-4c0f-895c-42ceb3201158 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:24:33.579237 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d48ed824-3027-4c0f-895c-42ceb3201158 admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:24:33.593386 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a9e5dc-0213-42de-affa-d49f80cdf6ad service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8191d3ccb874529b32f8d0026db629d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '540a32ef88764e8c8d9201ae22f6ddaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:24:33.598479 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a9e5dc-0213-42de-affa-d49f80cdf6ad service cinder] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:33.598896 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a9e5dc-0213-42de-affa-d49f80cdf6ad service cinder] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:33.601241 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a9e5dc-0213-42de-affa-d49f80cdf6ad service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:33.645158 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a9e5dc-0213-42de-affa-d49f80cdf6ad service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:24:33.663528 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a9e5dc-0213-42de-affa-d49f80cdf6ad service cinder] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:24:33.700090 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 878/1783] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 12:24:33 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 12:24:33.854378 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453b8829-eee5-4527-892d-8fa39ae27dd0 None None] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:24:33.871442 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453b8829-eee5-4527-892d-8fa39ae27dd0 None None] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:24:33.894217 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192596fa-b145-424b-985c-0f6a5c7516e5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8191d3ccb874529b32f8d0026db629d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '540a32ef88764e8c8d9201ae22f6ddaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:24:33.894863 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192596fa-b145-424b-985c-0f6a5c7516e5 service cinder] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:33.895148 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192596fa-b145-424b-985c-0f6a5c7516e5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:33.895420 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192596fa-b145-424b-985c-0f6a5c7516e5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:33.913305 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a9e5dc-0213-42de-affa-d49f80cdf6ad service cinder] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:24:33.920094 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-83a9e5dc-0213-42de-affa-d49f80cdf6ad service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:24:34.191611 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192596fa-b145-424b-985c-0f6a5c7516e5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:24:34.193109 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192596fa-b145-424b-985c-0f6a5c7516e5 service cinder] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:24:34.220548 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:24:34.240941 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a9e5dc-0213-42de-affa-d49f80cdf6ad service cinder] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:24:34.246096 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 906/1784] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 12:24:33 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 15 12:24:34.248066 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94494622-e556-4fa9-abaa-27a7f72b0af3 None None] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:34.248354 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94494622-e556-4fa9-abaa-27a7f72b0af3 None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:34.248625 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94494622-e556-4fa9-abaa-27a7f72b0af3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:34.252160 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f46a349-5be7-468c-982a-eccc6bc82e6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:24:34.252985 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f46a349-5be7-468c-982a-eccc6bc82e6d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:34.253151 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f46a349-5be7-468c-982a-eccc6bc82e6d admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:34.253424 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f46a349-5be7-468c-982a-eccc6bc82e6d admin admin] PATH_INFO: `/v3/projects/ef1b2914f1a64c2e99661187047fbd1f` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:34.258930 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f46a349-5be7-468c-982a-eccc6bc82e6d admin admin] RBAC: Authorizing `identity:delete_project(project_id=ef1b2914f1a64c2e99661187047fbd1f)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:24:34.270068 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f46a349-5be7-468c-982a-eccc6bc82e6d admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:24:34.328742 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.auth.core [None req-94494622-e556-4fa9-abaa-27a7f72b0af3 None None] MFA Rules not processed for user `4ead4be9666942f2b81cbb0f9ab69325`. Rule list: `[]` (Enabled: `True`). {{(pid=81449) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:24:34.333647 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-94494622-e556-4fa9-abaa-27a7f72b0af3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:24:34.490123 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-2f46a349-5be7-468c-982a-eccc6bc82e6d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ef1b2914f1a64c2e99661187047fbd1f', 'request_id': 'req-2f46a349-5be7-468c-982a-eccc6bc82e6d'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:24:34.541825 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-2f46a349-5be7-468c-982a-eccc6bc82e6d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ef1b2914f1a64c2e99661187047fbd1f', 'request_id': 'req-2f46a349-5be7-468c-982a-eccc6bc82e6d'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:24:34.577455 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-2f46a349-5be7-468c-982a-eccc6bc82e6d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ef1b2914f1a64c2e99661187047fbd1f', 'request_id': 'req-2f46a349-5be7-468c-982a-eccc6bc82e6d'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:24:34.610799 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 907/1785] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 12:24:34 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 364 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:24:34.637351 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-2f46a349-5be7-468c-982a-eccc6bc82e6d admin admin] The token cache is being invalidate because project ef1b2914f1a64c2e99661187047fbd1f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81448) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 12:24:34.637653 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-2f46a349-5be7-468c-982a-eccc6bc82e6d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:24:34.640855 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-2f46a349-5be7-468c-982a-eccc6bc82e6d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:24:34.654526 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 879/1786] 10.222.0.15 () {58 vars in 1233 bytes} [Thu Sep 15 12:24:33 2022] DELETE /identity/v3/projects/ef1b2914f1a64c2e99661187047fbd1f => generated 0 bytes in 742 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:24:34.662878 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94494622-e556-4fa9-abaa-27a7f72b0af3 None None] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:24:34.719086 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94494622-e556-4fa9-abaa-27a7f72b0af3 None None] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:24:34.818553 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0410af6e-5e03-4d78-9803-7ebf7ff26fde None None] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:34.818843 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0410af6e-5e03-4d78-9803-7ebf7ff26fde None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:34.819163 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0410af6e-5e03-4d78-9803-7ebf7ff26fde None None] PATH_INFO: `/` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:34.822075 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 880/1787] 10.222.0.15 () {54 vars in 912 bytes} [Thu Sep 15 12:24:34 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:24:34.828960 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4f93c8-ac2d-4ac9-8a32-8882bbf6ff09 None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:34.829321 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4f93c8-ac2d-4ac9-8a32-8882bbf6ff09 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:34.829509 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d4f93c8-ac2d-4ac9-8a32-8882bbf6ff09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:34.912341 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-2d4f93c8-ac2d-4ac9-8a32-8882bbf6ff09 None None] MFA Rules not processed for user `bfbb4d6809064f9cb9e06626ed7d6814`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:24:34.914110 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-2d4f93c8-ac2d-4ac9-8a32-8882bbf6ff09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:24:34.946814 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:24:34.967376 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e79613-da59-44e5-9df9-0a432b0e2a45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:24:34.970090 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e79613-da59-44e5-9df9-0a432b0e2a45 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:34.972444 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e79613-da59-44e5-9df9-0a432b0e2a45 admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:34.972920 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e79613-da59-44e5-9df9-0a432b0e2a45 admin admin] PATH_INFO: `/v3/users/ad4503ea3b454d0a8bd7bb7ee35646e5` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:34.977602 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e79613-da59-44e5-9df9-0a432b0e2a45 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ad4503ea3b454d0a8bd7bb7ee35646e5)` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:24:34.984849 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e79613-da59-44e5-9df9-0a432b0e2a45 admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:24:35.102012 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-c2e79613-da59-44e5-9df9-0a432b0e2a45 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ad4503ea3b454d0a8bd7bb7ee35646e5'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:24:35.130321 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-c2e79613-da59-44e5-9df9-0a432b0e2a45 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ad4503ea3b454d0a8bd7bb7ee35646e5'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:24:35.152897 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 881/1788] 10.222.0.15 () {58 vars in 1009 bytes} [Thu Sep 15 12:24:34 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 325 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 12:24:35.154876 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4f93c8-ac2d-4ac9-8a32-8882bbf6ff09 None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:24:35.157381 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-c2e79613-da59-44e5-9df9-0a432b0e2a45 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ad4503ea3b454d0a8bd7bb7ee35646e5'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:24:35.158196 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-c2e79613-da59-44e5-9df9-0a432b0e2a45 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ad4503ea3b454d0a8bd7bb7ee35646e5'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:24:35.162819 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-2d4f93c8-ac2d-4ac9-8a32-8882bbf6ff09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:24:35.187643 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-c2e79613-da59-44e5-9df9-0a432b0e2a45 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ad4503ea3b454d0a8bd7bb7ee35646e5'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:24:35.206236 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 908/1789] 10.222.0.15 () {58 vars in 1224 bytes} [Thu Sep 15 12:24:34 2022] DELETE /identity/v3/users/ad4503ea3b454d0a8bd7bb7ee35646e5 => generated 0 bytes in 544 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:24:35.235024 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d4f93c8-ac2d-4ac9-8a32-8882bbf6ff09 None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:24:35.273731 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e79613-da59-44e5-9df9-0a432b0e2a45 admin admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:24:35.275735 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-c2e79613-da59-44e5-9df9-0a432b0e2a45 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:24:35.394983 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:24:35.521584 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5b50b2-3207-4d06-bf68-e7b193892459 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:24:35.523114 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5b50b2-3207-4d06-bf68-e7b193892459 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:35.524408 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5b50b2-3207-4d06-bf68-e7b193892459 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:35.524927 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5b50b2-3207-4d06-bf68-e7b193892459 admin admin] PATH_INFO: `/v3/users/6b2c7e7a7d514af89e9c1232e1672445` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:35.529151 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e79613-da59-44e5-9df9-0a432b0e2a45 admin admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:24:35.530281 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5b50b2-3207-4d06-bf68-e7b193892459 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6b2c7e7a7d514af89e9c1232e1672445)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:24:35.534129 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5b50b2-3207-4d06-bf68-e7b193892459 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:24:35.547359 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33596ab7-0e7b-4e54-a708-42dd28bda0ba service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfbb4d6809064f9cb9e06626ed7d6814', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '540a32ef88764e8c8d9201ae22f6ddaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:24:35.547359 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33596ab7-0e7b-4e54-a708-42dd28bda0ba service neutron] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:35.547359 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33596ab7-0e7b-4e54-a708-42dd28bda0ba service neutron] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:35.547359 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33596ab7-0e7b-4e54-a708-42dd28bda0ba service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:35.548686 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-33596ab7-0e7b-4e54-a708-42dd28bda0ba service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:24:35.581742 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33596ab7-0e7b-4e54-a708-42dd28bda0ba service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:24:35.583231 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33596ab7-0e7b-4e54-a708-42dd28bda0ba service neutron] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:24:35.601359 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 909/1790] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 12:24:35 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 12:24:35.750739 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-fb5b50b2-3207-4d06-bf68-e7b193892459 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6b2c7e7a7d514af89e9c1232e1672445'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:24:35.804779 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-fb5b50b2-3207-4d06-bf68-e7b193892459 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6b2c7e7a7d514af89e9c1232e1672445'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:24:35.848259 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-fb5b50b2-3207-4d06-bf68-e7b193892459 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6b2c7e7a7d514af89e9c1232e1672445'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:24:35.849921 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-fb5b50b2-3207-4d06-bf68-e7b193892459 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6b2c7e7a7d514af89e9c1232e1672445'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:24:35.878982 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-fb5b50b2-3207-4d06-bf68-e7b193892459 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6b2c7e7a7d514af89e9c1232e1672445'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:24:35.914463 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 882/1791] 10.222.0.15 () {58 vars in 1224 bytes} [Thu Sep 15 12:24:35 2022] DELETE /identity/v3/users/6b2c7e7a7d514af89e9c1232e1672445 => generated 0 bytes in 761 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:24:35.919245 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5ed6a1-2bb7-40a4-b649-2e1fe6f58ba8 None None] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:35.919544 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5ed6a1-2bb7-40a4-b649-2e1fe6f58ba8 None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:35.919808 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5ed6a1-2bb7-40a4-b649-2e1fe6f58ba8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:35.995892 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.auth.core [None req-9a5ed6a1-2bb7-40a4-b649-2e1fe6f58ba8 None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81449) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:24:35.998199 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-9a5ed6a1-2bb7-40a4-b649-2e1fe6f58ba8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:24:36.109420 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e03c2f3-0742-4220-a58e-a2fa01346a19 None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:36.110106 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e03c2f3-0742-4220-a58e-a2fa01346a19 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:36.110542 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e03c2f3-0742-4220-a58e-a2fa01346a19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:36.154883 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 910/1792] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 12:24:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:24:36.235758 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-2e03c2f3-0742-4220-a58e-a2fa01346a19 None None] MFA Rules not processed for user `5c5f1c7f322d4ea8a7f62005f4296d64`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:24:36.237093 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-2e03c2f3-0742-4220-a58e-a2fa01346a19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:24:36.296900 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:24:36.385854 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5ed6a1-2bb7-40a4-b649-2e1fe6f58ba8 None None] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:24:36.387506 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-9a5ed6a1-2bb7-40a4-b649-2e1fe6f58ba8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:24:36.438657 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 883/1793] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 12:24:36 2022] POST /identity/v3/auth/tokens => generated 3280 bytes in 330 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:24:36.442964 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e03c2f3-0742-4220-a58e-a2fa01346a19 None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:24:36.444561 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-2e03c2f3-0742-4220-a58e-a2fa01346a19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:24:36.475780 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e03c2f3-0742-4220-a58e-a2fa01346a19 None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:24:36.489430 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2ca55f-fced-4c22-b3b6-22f3e10d4948 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:24:36.490344 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2ca55f-fced-4c22-b3b6-22f3e10d4948 admin admin] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:36.490793 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2ca55f-fced-4c22-b3b6-22f3e10d4948 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:36.491429 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2ca55f-fced-4c22-b3b6-22f3e10d4948 admin admin] PATH_INFO: `/v3/projects` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:36.493549 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2ca55f-fced-4c22-b3b6-22f3e10d4948 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:24:36.516300 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2ca55f-fced-4c22-b3b6-22f3e10d4948 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:24:36.617498 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5ed6a1-2bb7-40a4-b649-2e1fe6f58ba8 None None] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:24:36.647912 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318ab6bd-5a0c-4668-a2b5-72236c3e9948 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfbb4d6809064f9cb9e06626ed7d6814', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '540a32ef88764e8c8d9201ae22f6ddaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:24:36.648557 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318ab6bd-5a0c-4668-a2b5-72236c3e9948 service neutron] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:36.648939 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318ab6bd-5a0c-4668-a2b5-72236c3e9948 service neutron] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:36.655960 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318ab6bd-5a0c-4668-a2b5-72236c3e9948 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:36.677239 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 884/1794] 10.222.0.15 () {60 vars in 1153 bytes} [Thu Sep 15 12:24:36 2022] POST /identity/v3/projects => generated 403 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 12:24:36.678767 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2ca55f-fced-4c22-b3b6-22f3e10d4948 admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:24:36.705707 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318ab6bd-5a0c-4668-a2b5-72236c3e9948 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:24:36.708017 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318ab6bd-5a0c-4668-a2b5-72236c3e9948 service neutron] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:24:36.742714 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 911/1795] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 12:24:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 12:24:36.986758 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2ca55f-fced-4c22-b3b6-22f3e10d4948 admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:24:37.026461 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919f77c7-19b0-42a9-9bb1-312e421ee79e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:24:37.031939 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919f77c7-19b0-42a9-9bb1-312e421ee79e admin admin] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:37.032242 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919f77c7-19b0-42a9-9bb1-312e421ee79e admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:37.034071 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919f77c7-19b0-42a9-9bb1-312e421ee79e admin admin] PATH_INFO: `/v3/users` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:37.039876 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919f77c7-19b0-42a9-9bb1-312e421ee79e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:24:37.061299 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919f77c7-19b0-42a9-9bb1-312e421ee79e admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:24:37.074300 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd0c8cb-d628-4761-a0f3-68e4a7ad02e0 None None] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:37.074780 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd0c8cb-d628-4761-a0f3-68e4a7ad02e0 None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:37.075741 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd0c8cb-d628-4761-a0f3-68e4a7ad02e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:37.156020 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.auth.core [None req-ddd0c8cb-d628-4761-a0f3-68e4a7ad02e0 None None] MFA Rules not processed for user `bfbb4d6809064f9cb9e06626ed7d6814`. Rule list: `[]` (Enabled: `True`). {{(pid=81449) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:24:37.163048 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-ddd0c8cb-d628-4761-a0f3-68e4a7ad02e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:24:37.224744 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 885/1796] 10.222.0.15 () {60 vars in 1144 bytes} [Thu Sep 15 12:24:36 2022] POST /identity/v3/users => generated 308 bytes in 544 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 12:24:37.224744 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919f77c7-19b0-42a9-9bb1-312e421ee79e admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:24:37.233497 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-919f77c7-19b0-42a9-9bb1-312e421ee79e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:24:37.303403 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919f77c7-19b0-42a9-9bb1-312e421ee79e admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:24:37.374261 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca5c9cb-9034-4051-b071-a92da87e18ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:24:37.378787 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca5c9cb-9034-4051-b071-a92da87e18ea admin admin] REQUEST_METHOD: `DELETE` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:37.389226 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca5c9cb-9034-4051-b071-a92da87e18ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:37.392348 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca5c9cb-9034-4051-b071-a92da87e18ea admin admin] PATH_INFO: `/v3/projects/c461164ac5f64ca49d822a8f5b2e6b12` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:37.401861 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca5c9cb-9034-4051-b071-a92da87e18ea admin admin] RBAC: Authorizing `identity:delete_project(project_id=c461164ac5f64ca49d822a8f5b2e6b12)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:24:37.409039 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 912/1797] 10.222.0.26 () {58 vars in 1015 bytes} [Thu Sep 15 12:24:37 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 336 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:24:37.410016 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd0c8cb-d628-4761-a0f3-68e4a7ad02e0 None None] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:24:37.421266 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca5c9cb-9034-4051-b071-a92da87e18ea admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:24:37.471243 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd0c8cb-d628-4761-a0f3-68e4a7ad02e0 None None] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:24:37.495999 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8de46a5-7ef7-40e4-8dd1-b2cc9dcb46b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:24:37.496732 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8de46a5-7ef7-40e4-8dd1-b2cc9dcb46b0 admin admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:37.497117 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8de46a5-7ef7-40e4-8dd1-b2cc9dcb46b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:37.497395 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8de46a5-7ef7-40e4-8dd1-b2cc9dcb46b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:37.498105 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8de46a5-7ef7-40e4-8dd1-b2cc9dcb46b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:24:37.499837 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8de46a5-7ef7-40e4-8dd1-b2cc9dcb46b0 admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:24:37.572436 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 913/1798] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 12:24:37 2022] GET /identity/v3/roles => generated 1420 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 12:24:37.575829 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8de46a5-7ef7-40e4-8dd1-b2cc9dcb46b0 admin admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:24:37.582816 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-a8de46a5-7ef7-40e4-8dd1-b2cc9dcb46b0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:24:37.631488 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8de46a5-7ef7-40e4-8dd1-b2cc9dcb46b0 admin admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:24:37.643562 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:24:37.695945 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-bca5c9cb-9034-4051-b071-a92da87e18ea admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c461164ac5f64ca49d822a8f5b2e6b12', 'request_id': 'req-bca5c9cb-9034-4051-b071-a92da87e18ea'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:24:37.737251 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-bca5c9cb-9034-4051-b071-a92da87e18ea admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c461164ac5f64ca49d822a8f5b2e6b12', 'request_id': 'req-bca5c9cb-9034-4051-b071-a92da87e18ea'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:24:37.749410 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-bca5c9cb-9034-4051-b071-a92da87e18ea admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c461164ac5f64ca49d822a8f5b2e6b12', 'request_id': 'req-bca5c9cb-9034-4051-b071-a92da87e18ea'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:24:37.781796 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a1ad79-0d68-414d-a0b7-0d116fec98b0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfbb4d6809064f9cb9e06626ed7d6814', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '540a32ef88764e8c8d9201ae22f6ddaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:24:37.784324 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a1ad79-0d68-414d-a0b7-0d116fec98b0 service neutron] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:37.789275 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a1ad79-0d68-414d-a0b7-0d116fec98b0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:37.790961 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a1ad79-0d68-414d-a0b7-0d116fec98b0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:37.811213 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a1ad79-0d68-414d-a0b7-0d116fec98b0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:24:37.812202 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-bca5c9cb-9034-4051-b071-a92da87e18ea admin admin] The token cache is being invalidate because project c461164ac5f64ca49d822a8f5b2e6b12 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81448) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 12:24:37.812500 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-bca5c9cb-9034-4051-b071-a92da87e18ea admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:24:37.814662 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-bca5c9cb-9034-4051-b071-a92da87e18ea admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:24:37.815448 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a1ad79-0d68-414d-a0b7-0d116fec98b0 service neutron] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:24:37.816848 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 886/1799] 10.222.0.15 () {58 vars in 1233 bytes} [Thu Sep 15 12:24:37 2022] DELETE /identity/v3/projects/c461164ac5f64ca49d822a8f5b2e6b12 => generated 0 bytes in 594 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:24:37.818823 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca5c9cb-9034-4051-b071-a92da87e18ea admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:24:37.820371 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-bca5c9cb-9034-4051-b071-a92da87e18ea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:24:37.845267 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 914/1800] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 12:24:37 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 272 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 12:24:37.847729 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a49b563-591a-4bc5-9d60-104262b47913 None None] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:37.849185 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a49b563-591a-4bc5-9d60-104262b47913 None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:37.851126 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a49b563-591a-4bc5-9d60-104262b47913 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:37.963175 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:24:37.965830 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.auth.core [None req-7a49b563-591a-4bc5-9d60-104262b47913 None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81449) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:24:37.973724 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-7a49b563-591a-4bc5-9d60-104262b47913 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:24:38.111008 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca5c9cb-9034-4051-b071-a92da87e18ea admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:24:38.121517 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc4994b-0118-4197-83ef-ede63326c4ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:24:38.122221 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc4994b-0118-4197-83ef-ede63326c4ed admin admin] REQUEST_METHOD: `PUT` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:38.122506 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc4994b-0118-4197-83ef-ede63326c4ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:38.122942 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc4994b-0118-4197-83ef-ede63326c4ed admin admin] PATH_INFO: `/v3/projects/d018f19bc21a4471ba7577551ed60dfe/users/3976c4c8f9704a1db410926eb62fa31e/roles/9c769b5c82884766b2c510b1f0f1364c` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:38.159559 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 915/1801] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 12:24:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 314 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:24:38.168057 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e847d22f-c8b9-4542-b936-7959ce51a60b None None] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:38.168580 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e847d22f-c8b9-4542-b936-7959ce51a60b None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:38.169042 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e847d22f-c8b9-4542-b936-7959ce51a60b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:38.193977 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc4994b-0118-4197-83ef-ede63326c4ed admin admin] RBAC: Authorizing `identity:create_grant(project_id=d018f19bc21a4471ba7577551ed60dfe, user_id=3976c4c8f9704a1db410926eb62fa31e, role_id=9c769b5c82884766b2c510b1f0f1364c)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:24:38.203127 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc4994b-0118-4197-83ef-ede63326c4ed admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:24:38.241578 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 887/1802] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 15 12:24:37 2022] PUT /identity/v3/projects/d018f19bc21a4471ba7577551ed60dfe/users/3976c4c8f9704a1db410926eb62fa31e/roles/9c769b5c82884766b2c510b1f0f1364c => generated 0 bytes in 420 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:24:38.243067 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc4994b-0118-4197-83ef-ede63326c4ed admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:24:38.246868 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.auth.core [None req-e847d22f-c8b9-4542-b936-7959ce51a60b None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81449) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:24:38.266972 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-e847d22f-c8b9-4542-b936-7959ce51a60b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:24:38.506270 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 916/1803] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 12:24:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 339 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 12:24:38.511080 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e847d22f-c8b9-4542-b936-7959ce51a60b None None] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:24:38.533133 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc4994b-0118-4197-83ef-ede63326c4ed admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:24:38.539426 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e847d22f-c8b9-4542-b936-7959ce51a60b None None] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:24:38.560740 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433822cd-7695-4518-beee-6673f44331b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:24:38.561424 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433822cd-7695-4518-beee-6673f44331b2 admin admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:38.561779 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433822cd-7695-4518-beee-6673f44331b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:38.562058 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433822cd-7695-4518-beee-6673f44331b2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:38.562718 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-433822cd-7695-4518-beee-6673f44331b2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81449) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:24:38.563023 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-433822cd-7695-4518-beee-6673f44331b2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:24:38.564210 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b81c1c-1c5f-4658-b7b9-61d516947574 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:24:38.564621 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-433822cd-7695-4518-beee-6673f44331b2 admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:24:38.565430 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b81c1c-1c5f-4658-b7b9-61d516947574 admin admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:38.565831 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b81c1c-1c5f-4658-b7b9-61d516947574 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:38.566333 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b81c1c-1c5f-4658-b7b9-61d516947574 admin admin] PATH_INFO: `/v3/roles` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:38.569382 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b81c1c-1c5f-4658-b7b9-61d516947574 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:24:38.574226 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b81c1c-1c5f-4658-b7b9-61d516947574 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:24:38.608238 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 917/1804] 10.222.0.15 () {58 vars in 1153 bytes} [Thu Sep 15 12:24:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 12:24:38.612713 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433822cd-7695-4518-beee-6673f44331b2 admin admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:24:38.638683 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 888/1805] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 12:24:38 2022] GET /identity/v3/roles => generated 1420 bytes in 395 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 15 12:24:38.650826 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433822cd-7695-4518-beee-6673f44331b2 admin admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:24:38.658797 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b81c1c-1c5f-4658-b7b9-61d516947574 admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:24:38.670630 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992f82b5-01bb-44e8-9622-6bb55e0694ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:24:38.671398 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992f82b5-01bb-44e8-9622-6bb55e0694ae admin admin] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:38.671788 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992f82b5-01bb-44e8-9622-6bb55e0694ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:38.672885 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992f82b5-01bb-44e8-9622-6bb55e0694ae admin admin] PATH_INFO: `/v3/projects` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:38.673647 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992f82b5-01bb-44e8-9622-6bb55e0694ae admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:24:38.678827 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992f82b5-01bb-44e8-9622-6bb55e0694ae admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:24:38.713524 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b81c1c-1c5f-4658-b7b9-61d516947574 admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:24:38.742305 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3faf16-d13d-4e69-9fbf-365f89c2b851 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:24:38.748683 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3faf16-d13d-4e69-9fbf-365f89c2b851 admin admin] REQUEST_METHOD: `PUT` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:38.753262 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3faf16-d13d-4e69-9fbf-365f89c2b851 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:38.757164 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3faf16-d13d-4e69-9fbf-365f89c2b851 admin admin] PATH_INFO: `/v3/projects/d018f19bc21a4471ba7577551ed60dfe/users/3976c4c8f9704a1db410926eb62fa31e/roles/9c769b5c82884766b2c510b1f0f1364c` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:38.766094 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3faf16-d13d-4e69-9fbf-365f89c2b851 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d018f19bc21a4471ba7577551ed60dfe, user_id=3976c4c8f9704a1db410926eb62fa31e, role_id=9c769b5c82884766b2c510b1f0f1364c)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:24:38.776280 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 918/1806] 10.222.0.15 () {60 vars in 1153 bytes} [Thu Sep 15 12:24:38 2022] POST /identity/v3/projects => generated 375 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 12:24:38.779694 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992f82b5-01bb-44e8-9622-6bb55e0694ae admin admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:24:38.781671 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3faf16-d13d-4e69-9fbf-365f89c2b851 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:24:38.791142 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-992f82b5-01bb-44e8-9622-6bb55e0694ae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:24:38.840053 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 889/1807] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 15 12:24:38 2022] PUT /identity/v3/projects/d018f19bc21a4471ba7577551ed60dfe/users/3976c4c8f9704a1db410926eb62fa31e/roles/9c769b5c82884766b2c510b1f0f1364c => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:24:38.847480 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3faf16-d13d-4e69-9fbf-365f89c2b851 admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:24:38.862471 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:24:39.010447 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992f82b5-01bb-44e8-9622-6bb55e0694ae admin admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:24:39.115330 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3faf16-d13d-4e69-9fbf-365f89c2b851 admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:24:39.134989 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f10fb4-784b-4a6e-82bf-587b65ad0c61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:24:39.136092 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f10fb4-784b-4a6e-82bf-587b65ad0c61 admin admin] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:39.136453 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f10fb4-784b-4a6e-82bf-587b65ad0c61 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:39.136718 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f10fb4-784b-4a6e-82bf-587b65ad0c61 admin admin] PATH_INFO: `/v3/users` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:39.137492 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f10fb4-784b-4a6e-82bf-587b65ad0c61 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:24:39.152974 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f10fb4-784b-4a6e-82bf-587b65ad0c61 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:24:39.153501 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e778169-532e-4d7e-b9f4-7dc0751e72f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:24:39.154428 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e778169-532e-4d7e-b9f4-7dc0751e72f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:39.154994 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e778169-532e-4d7e-b9f4-7dc0751e72f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:39.155487 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e778169-532e-4d7e-b9f4-7dc0751e72f7 admin admin] PATH_INFO: `/v3/projects/b657bd5a9cfc4f7bb7130356dcf7966b` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:39.162326 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e778169-532e-4d7e-b9f4-7dc0751e72f7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b657bd5a9cfc4f7bb7130356dcf7966b)` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:24:39.167888 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e778169-532e-4d7e-b9f4-7dc0751e72f7 admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:24:39.219915 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 890/1808] 10.222.0.15 () {60 vars in 1144 bytes} [Thu Sep 15 12:24:38 2022] POST /identity/v3/users => generated 288 bytes in 379 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 12:24:39.221433 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f10fb4-784b-4a6e-82bf-587b65ad0c61 admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:24:39.269861 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f10fb4-784b-4a6e-82bf-587b65ad0c61 admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:24:39.286274 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f923f87c-93ca-42e4-acf7-94056da9a571 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:24:39.287862 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f923f87c-93ca-42e4-acf7-94056da9a571 admin admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:39.289045 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f923f87c-93ca-42e4-acf7-94056da9a571 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:39.320477 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f923f87c-93ca-42e4-acf7-94056da9a571 admin admin] PATH_INFO: `/v3/roles` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:39.322917 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f923f87c-93ca-42e4-acf7-94056da9a571 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:24:39.334333 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f923f87c-93ca-42e4-acf7-94056da9a571 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:24:39.376346 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 891/1809] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 12:24:39 2022] GET /identity/v3/roles => generated 1420 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 12:24:39.377108 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-3e778169-532e-4d7e-b9f4-7dc0751e72f7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b657bd5a9cfc4f7bb7130356dcf7966b', 'request_id': 'req-3e778169-532e-4d7e-b9f4-7dc0751e72f7'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:24:39.380988 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f923f87c-93ca-42e4-acf7-94056da9a571 admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:24:39.395114 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-3e778169-532e-4d7e-b9f4-7dc0751e72f7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b657bd5a9cfc4f7bb7130356dcf7966b', 'request_id': 'req-3e778169-532e-4d7e-b9f4-7dc0751e72f7'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:24:39.404518 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-3e778169-532e-4d7e-b9f4-7dc0751e72f7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b657bd5a9cfc4f7bb7130356dcf7966b', 'request_id': 'req-3e778169-532e-4d7e-b9f4-7dc0751e72f7'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:24:39.409978 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f923f87c-93ca-42e4-acf7-94056da9a571 admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:24:39.420467 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:24:39.455521 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-3e778169-532e-4d7e-b9f4-7dc0751e72f7 admin admin] The token cache is being invalidate because project b657bd5a9cfc4f7bb7130356dcf7966b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81449) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 12:24:39.459818 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-3e778169-532e-4d7e-b9f4-7dc0751e72f7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:24:39.460936 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-3e778169-532e-4d7e-b9f4-7dc0751e72f7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:24:39.466334 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 919/1810] 10.222.0.15 () {58 vars in 1233 bytes} [Thu Sep 15 12:24:38 2022] DELETE /identity/v3/projects/b657bd5a9cfc4f7bb7130356dcf7966b => generated 0 bytes in 689 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:24:39.473128 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e778169-532e-4d7e-b9f4-7dc0751e72f7 admin admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:24:39.484417 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-3e778169-532e-4d7e-b9f4-7dc0751e72f7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:24:39.616117 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:24:39.639125 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e778169-532e-4d7e-b9f4-7dc0751e72f7 admin admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:24:39.640842 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07090993-d25c-4bc9-8977-caa54ae93647 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:24:39.641500 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07090993-d25c-4bc9-8977-caa54ae93647 admin admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:39.641782 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07090993-d25c-4bc9-8977-caa54ae93647 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:39.642052 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07090993-d25c-4bc9-8977-caa54ae93647 admin admin] PATH_INFO: `/v3/roles` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:39.642752 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07090993-d25c-4bc9-8977-caa54ae93647 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:24:39.644745 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07090993-d25c-4bc9-8977-caa54ae93647 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:24:39.649006 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9afd1363-06cf-4d48-ac23-57903df03814 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:24:39.652986 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afd1363-06cf-4d48-ac23-57903df03814 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:39.654229 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afd1363-06cf-4d48-ac23-57903df03814 admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:39.654746 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9afd1363-06cf-4d48-ac23-57903df03814 admin admin] PATH_INFO: `/v3/users/68e37396900f49488ebc8b2a4459dea2` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:39.656637 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9afd1363-06cf-4d48-ac23-57903df03814 admin admin] RBAC: Authorizing `identity:delete_user(user_id=68e37396900f49488ebc8b2a4459dea2)` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:24:39.660345 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9afd1363-06cf-4d48-ac23-57903df03814 admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:24:39.671732 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 892/1811] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 12:24:39 2022] GET /identity/v3/roles => generated 1420 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 12:24:39.677876 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07090993-d25c-4bc9-8977-caa54ae93647 admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:24:39.689120 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-07090993-d25c-4bc9-8977-caa54ae93647 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:24:39.717904 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07090993-d25c-4bc9-8977-caa54ae93647 admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:24:39.733687 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae07157e-d257-487f-9ce9-e6977e883da8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:24:39.734468 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae07157e-d257-487f-9ce9-e6977e883da8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:39.734759 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae07157e-d257-487f-9ce9-e6977e883da8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:39.735123 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae07157e-d257-487f-9ce9-e6977e883da8 admin admin] PATH_INFO: `/v3/projects/68f00d468c394abcb256b3904d8ad3bf/users/ed5e8c2d770a42848f22ed2243f2a5b0/roles/0679d6628a25438d90dc4bdc0401eda2` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:39.739900 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-9afd1363-06cf-4d48-ac23-57903df03814 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '68e37396900f49488ebc8b2a4459dea2'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:24:39.751580 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-9afd1363-06cf-4d48-ac23-57903df03814 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '68e37396900f49488ebc8b2a4459dea2'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:24:39.771894 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae07157e-d257-487f-9ce9-e6977e883da8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=68f00d468c394abcb256b3904d8ad3bf, user_id=ed5e8c2d770a42848f22ed2243f2a5b0, role_id=0679d6628a25438d90dc4bdc0401eda2)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:24:39.775188 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae07157e-d257-487f-9ce9-e6977e883da8 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:24:39.777511 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-9afd1363-06cf-4d48-ac23-57903df03814 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '68e37396900f49488ebc8b2a4459dea2'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:24:39.779202 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-9afd1363-06cf-4d48-ac23-57903df03814 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '68e37396900f49488ebc8b2a4459dea2'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:24:39.796622 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-9afd1363-06cf-4d48-ac23-57903df03814 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '68e37396900f49488ebc8b2a4459dea2'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:24:39.805969 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 920/1812] 10.222.0.15 () {58 vars in 1224 bytes} [Thu Sep 15 12:24:39 2022] DELETE /identity/v3/users/68e37396900f49488ebc8b2a4459dea2 => generated 0 bytes in 336 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:24:39.825699 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 893/1813] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 15 12:24:39 2022] PUT /identity/v3/projects/68f00d468c394abcb256b3904d8ad3bf/users/ed5e8c2d770a42848f22ed2243f2a5b0/roles/0679d6628a25438d90dc4bdc0401eda2 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:24:39.828227 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58d4ff8-4861-4aea-99a1-d12859deb78a None None] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:39.828705 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58d4ff8-4861-4aea-99a1-d12859deb78a None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:39.829072 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58d4ff8-4861-4aea-99a1-d12859deb78a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:39.886776 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.auth.core [None req-f58d4ff8-4861-4aea-99a1-d12859deb78a None None] MFA Rules not processed for user `ed5e8c2d770a42848f22ed2243f2a5b0`. Rule list: `[]` (Enabled: `True`). {{(pid=81449) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:24:39.888384 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-f58d4ff8-4861-4aea-99a1-d12859deb78a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:24:40.016568 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 921/1814] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 12:24:39 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:24:40.028399 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae07157e-d257-487f-9ce9-e6977e883da8 admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:24:40.030819 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-ae07157e-d257-487f-9ce9-e6977e883da8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:24:40.187850 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae07157e-d257-487f-9ce9-e6977e883da8 admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:24:40.320088 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28188f69-75dd-4ced-991a-d2fb546a1099 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3aed8f78903f44cda9e5254aa46cf2b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '540a32ef88764e8c8d9201ae22f6ddaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:24:40.321108 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28188f69-75dd-4ced-991a-d2fb546a1099 service glance] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:40.321904 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28188f69-75dd-4ced-991a-d2fb546a1099 service glance] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:40.322370 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28188f69-75dd-4ced-991a-d2fb546a1099 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:40.339991 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28188f69-75dd-4ced-991a-d2fb546a1099 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:24:40.343051 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28188f69-75dd-4ced-991a-d2fb546a1099 service glance] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:24:40.356650 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 894/1815] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 12:24:40 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 12:24:40.410783 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58d4ff8-4861-4aea-99a1-d12859deb78a None None] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:24:40.412496 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-f58d4ff8-4861-4aea-99a1-d12859deb78a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:24:40.579827 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58d4ff8-4861-4aea-99a1-d12859deb78a None None] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:24:40.592443 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d595cf22-ea82-4031-a7aa-e3dde36659da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:24:40.593386 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d595cf22-ea82-4031-a7aa-e3dde36659da admin admin] REQUEST_METHOD: `DELETE` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:40.593731 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d595cf22-ea82-4031-a7aa-e3dde36659da admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:40.594094 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d595cf22-ea82-4031-a7aa-e3dde36659da admin admin] PATH_INFO: `/v3/projects/19e6ec5e336d48f1a10ab8d91c95b1a1` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:40.596731 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d595cf22-ea82-4031-a7aa-e3dde36659da admin admin] RBAC: Authorizing `identity:delete_project(project_id=19e6ec5e336d48f1a10ab8d91c95b1a1)` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:24:40.601666 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d595cf22-ea82-4031-a7aa-e3dde36659da admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:24:40.610045 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:24:40.805262 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-d595cf22-ea82-4031-a7aa-e3dde36659da admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '19e6ec5e336d48f1a10ab8d91c95b1a1', 'request_id': 'req-d595cf22-ea82-4031-a7aa-e3dde36659da'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:24:40.826060 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-d595cf22-ea82-4031-a7aa-e3dde36659da admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '19e6ec5e336d48f1a10ab8d91c95b1a1', 'request_id': 'req-d595cf22-ea82-4031-a7aa-e3dde36659da'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:24:40.847792 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-d595cf22-ea82-4031-a7aa-e3dde36659da admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '19e6ec5e336d48f1a10ab8d91c95b1a1', 'request_id': 'req-d595cf22-ea82-4031-a7aa-e3dde36659da'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:24:40.873285 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-d595cf22-ea82-4031-a7aa-e3dde36659da admin admin] The token cache is being invalidate because project 19e6ec5e336d48f1a10ab8d91c95b1a1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81449) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 12:24:40.875153 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-d595cf22-ea82-4031-a7aa-e3dde36659da admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:24:40.876662 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-d595cf22-ea82-4031-a7aa-e3dde36659da admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:24:40.881308 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 922/1816] 10.222.0.15 () {58 vars in 1233 bytes} [Thu Sep 15 12:24:40 2022] DELETE /identity/v3/projects/19e6ec5e336d48f1a10ab8d91c95b1a1 => generated 0 bytes in 471 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:24:40.885969 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd3b0bd-e6a6-4d6a-90f0-70641d2e1634 None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:40.886553 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd3b0bd-e6a6-4d6a-90f0-70641d2e1634 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:40.886553 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd3b0bd-e6a6-4d6a-90f0-70641d2e1634 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:40.952274 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-efd3b0bd-e6a6-4d6a-90f0-70641d2e1634 None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:24:40.959552 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-efd3b0bd-e6a6-4d6a-90f0-70641d2e1634 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:24:41.014808 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:24:41.205786 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 895/1817] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 12:24:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 321 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:24:41.208546 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e235f0b3-3487-4052-ad6a-8d82045fb1e8 None None] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:41.208925 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e235f0b3-3487-4052-ad6a-8d82045fb1e8 None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:41.209121 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e235f0b3-3487-4052-ad6a-8d82045fb1e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:41.302890 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.auth.core [None req-e235f0b3-3487-4052-ad6a-8d82045fb1e8 None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81449) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:24:41.304309 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-e235f0b3-3487-4052-ad6a-8d82045fb1e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:24:41.330750 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 923/1818] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 12:24:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:24:41.332280 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd3b0bd-e6a6-4d6a-90f0-70641d2e1634 None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:24:41.363125 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd3b0bd-e6a6-4d6a-90f0-70641d2e1634 None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:24:41.397572 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c6f105-a208-4310-941d-c6a931573ba1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:24:41.398219 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c6f105-a208-4310-941d-c6a931573ba1 admin admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:41.398499 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c6f105-a208-4310-941d-c6a931573ba1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:41.398781 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c6f105-a208-4310-941d-c6a931573ba1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:41.399430 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c6f105-a208-4310-941d-c6a931573ba1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:24:41.400292 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c6f105-a208-4310-941d-c6a931573ba1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:24:41.402194 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c6f105-a208-4310-941d-c6a931573ba1 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:24:41.441945 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 896/1819] 10.222.0.15 () {58 vars in 1153 bytes} [Thu Sep 15 12:24:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:24:41.444617 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e235f0b3-3487-4052-ad6a-8d82045fb1e8 None None] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:24:41.458554 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e235f0b3-3487-4052-ad6a-8d82045fb1e8 None None] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:24:41.470201 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032f1ceb-0df0-4517-9e64-b910fac9851b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:24:41.471341 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032f1ceb-0df0-4517-9e64-b910fac9851b admin admin] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:41.471794 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032f1ceb-0df0-4517-9e64-b910fac9851b admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:41.472250 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032f1ceb-0df0-4517-9e64-b910fac9851b admin admin] PATH_INFO: `/v3/projects` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:41.473124 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032f1ceb-0df0-4517-9e64-b910fac9851b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:24:41.475306 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032f1ceb-0df0-4517-9e64-b910fac9851b admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:24:41.537626 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 924/1820] 10.222.0.15 () {60 vars in 1153 bytes} [Thu Sep 15 12:24:41 2022] POST /identity/v3/projects => generated 365 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 12:24:41.541745 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c6f105-a208-4310-941d-c6a931573ba1 admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:24:41.558707 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c6f105-a208-4310-941d-c6a931573ba1 admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:24:41.810047 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bcd8b59-9700-4d44-b3aa-50186bc1009f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:24:41.818132 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcd8b59-9700-4d44-b3aa-50186bc1009f admin admin] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:41.819813 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcd8b59-9700-4d44-b3aa-50186bc1009f admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:41.828772 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bcd8b59-9700-4d44-b3aa-50186bc1009f admin admin] PATH_INFO: `/v3/users` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:41.831304 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bcd8b59-9700-4d44-b3aa-50186bc1009f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:24:41.848904 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bcd8b59-9700-4d44-b3aa-50186bc1009f admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:24:41.866558 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fa9e55-43f1-4e7c-9ada-b3a50d195de1 None None] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:41.867663 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fa9e55-43f1-4e7c-9ada-b3a50d195de1 None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:41.868126 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fa9e55-43f1-4e7c-9ada-b3a50d195de1 None None] PATH_INFO: `/` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:41.870506 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 925/1821] 10.222.0.15 () {54 vars in 921 bytes} [Thu Sep 15 12:24:41 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:24:41.893918 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28ba95d-13e7-4ba6-9085-cc7ab826938f None None] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:41.894431 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28ba95d-13e7-4ba6-9085-cc7ab826938f None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:41.894848 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28ba95d-13e7-4ba6-9085-cc7ab826938f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:41.910163 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 897/1822] 10.222.0.15 () {60 vars in 1144 bytes} [Thu Sep 15 12:24:41 2022] POST /identity/v3/users => generated 283 bytes in 369 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 12:24:41.914203 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bcd8b59-9700-4d44-b3aa-50186bc1009f admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:24:41.942232 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bcd8b59-9700-4d44-b3aa-50186bc1009f admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:24:41.952318 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.auth.core [None req-a28ba95d-13e7-4ba6-9085-cc7ab826938f None None] MFA Rules not processed for user `eface89e261840a2abb4206be26455c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81449) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:24:41.961930 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-a28ba95d-13e7-4ba6-9085-cc7ab826938f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:24:41.962579 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44df8dff-f7e2-4b60-b888-f3de25c3d99b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:24:41.963691 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44df8dff-f7e2-4b60-b888-f3de25c3d99b admin admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:41.964610 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44df8dff-f7e2-4b60-b888-f3de25c3d99b admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:41.965237 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44df8dff-f7e2-4b60-b888-f3de25c3d99b admin admin] PATH_INFO: `/v3/roles` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:41.966413 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44df8dff-f7e2-4b60-b888-f3de25c3d99b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:24:41.968665 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44df8dff-f7e2-4b60-b888-f3de25c3d99b admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:24:42.022585 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 898/1823] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 12:24:41 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 15 12:24:42.028237 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44df8dff-f7e2-4b60-b888-f3de25c3d99b admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:24:42.053418 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44df8dff-f7e2-4b60-b888-f3de25c3d99b admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:24:42.071315 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220f4e26-49f5-40b1-8d33-6d05380c8e2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:24:42.072206 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220f4e26-49f5-40b1-8d33-6d05380c8e2d admin admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:42.072657 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220f4e26-49f5-40b1-8d33-6d05380c8e2d admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:42.073103 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220f4e26-49f5-40b1-8d33-6d05380c8e2d admin admin] PATH_INFO: `/v3/roles` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:42.075183 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220f4e26-49f5-40b1-8d33-6d05380c8e2d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:24:42.076979 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220f4e26-49f5-40b1-8d33-6d05380c8e2d admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:24:42.122073 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 899/1824] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 12:24:42 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 12:24:42.132992 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5964c51e-9b57-42fb-b742-a8d9e1c75863 None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:42.134650 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5964c51e-9b57-42fb-b742-a8d9e1c75863 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:42.135215 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5964c51e-9b57-42fb-b742-a8d9e1c75863 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:42.146612 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 926/1825] 10.222.0.15 () {58 vars in 1018 bytes} [Thu Sep 15 12:24:41 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 255 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:24:42.148886 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28ba95d-13e7-4ba6-9085-cc7ab826938f None None] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:24:42.164211 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28ba95d-13e7-4ba6-9085-cc7ab826938f None None] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:24:42.170999 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7a2a61-df1c-4fc3-9d69-ec33cadc77ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:24:42.172070 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7a2a61-df1c-4fc3-9d69-ec33cadc77ea admin admin] REQUEST_METHOD: `PUT` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:42.178365 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7a2a61-df1c-4fc3-9d69-ec33cadc77ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:42.179543 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7a2a61-df1c-4fc3-9d69-ec33cadc77ea admin admin] PATH_INFO: `/v3/projects/927baf46d46845208f83d9b89e1eb6d3/users/fca9eb3e04884525b788369196668d2b/roles/0679d6628a25438d90dc4bdc0401eda2` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:42.202913 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f7a2a61-df1c-4fc3-9d69-ec33cadc77ea admin admin] RBAC: Authorizing `identity:create_grant(project_id=927baf46d46845208f83d9b89e1eb6d3, user_id=fca9eb3e04884525b788369196668d2b, role_id=0679d6628a25438d90dc4bdc0401eda2)` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:24:42.206729 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f7a2a61-df1c-4fc3-9d69-ec33cadc77ea admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:24:42.239339 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 927/1826] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 15 12:24:42 2022] PUT /identity/v3/projects/927baf46d46845208f83d9b89e1eb6d3/users/fca9eb3e04884525b788369196668d2b/roles/0679d6628a25438d90dc4bdc0401eda2 => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:24:42.250297 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62dac8e0-f984-4fdc-abd6-ce6b004ba9ff None None] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:42.250861 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62dac8e0-f984-4fdc-abd6-ce6b004ba9ff None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:42.251247 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62dac8e0-f984-4fdc-abd6-ce6b004ba9ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:42.268542 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-5964c51e-9b57-42fb-b742-a8d9e1c75863 None None] MFA Rules not processed for user `3976c4c8f9704a1db410926eb62fa31e`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:24:42.271685 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-5964c51e-9b57-42fb-b742-a8d9e1c75863 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:24:42.312199 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.auth.core [None req-62dac8e0-f984-4fdc-abd6-ce6b004ba9ff None None] MFA Rules not processed for user `fca9eb3e04884525b788369196668d2b`. Rule list: `[]` (Enabled: `True`). {{(pid=81449) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:24:42.313735 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-62dac8e0-f984-4fdc-abd6-ce6b004ba9ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:24:42.336679 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:24:42.400061 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:24:42.403563 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 928/1827] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 12:24:42 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 12:24:42.404503 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62dac8e0-f984-4fdc-abd6-ce6b004ba9ff None None] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:24:42.405978 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-62dac8e0-f984-4fdc-abd6-ce6b004ba9ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:24:42.408802 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 900/1828] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 12:24:42 2022] POST /identity/v3/auth/tokens => generated 3347 bytes in 285 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:24:42.409606 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5964c51e-9b57-42fb-b742-a8d9e1c75863 None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:24:42.485164 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5964c51e-9b57-42fb-b742-a8d9e1c75863 None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:24:42.491093 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24dfcf9a-2399-48dc-adc5-35fc67832b3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:24:42.491882 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24dfcf9a-2399-48dc-adc5-35fc67832b3a admin admin] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:42.492165 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24dfcf9a-2399-48dc-adc5-35fc67832b3a admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:42.492434 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24dfcf9a-2399-48dc-adc5-35fc67832b3a admin admin] PATH_INFO: `/v3/projects` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:42.493121 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24dfcf9a-2399-48dc-adc5-35fc67832b3a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:24:42.494765 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62dac8e0-f984-4fdc-abd6-ce6b004ba9ff None None] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:24:42.495172 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24dfcf9a-2399-48dc-adc5-35fc67832b3a admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:24:42.502508 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0170f479-93b8-4997-8d38-ae87e8c8282d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eface89e261840a2abb4206be26455c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '540a32ef88764e8c8d9201ae22f6ddaf', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:24:42.503387 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0170f479-93b8-4997-8d38-ae87e8c8282d service nova] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:42.503772 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0170f479-93b8-4997-8d38-ae87e8c8282d service nova] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:42.504147 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0170f479-93b8-4997-8d38-ae87e8c8282d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:42.514060 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0170f479-93b8-4997-8d38-ae87e8c8282d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:24:42.516051 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0170f479-93b8-4997-8d38-ae87e8c8282d service nova] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:24:42.529294 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 901/1829] 10.222.0.15 () {60 vars in 1153 bytes} [Thu Sep 15 12:24:42 2022] POST /identity/v3/projects => generated 365 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 12:24:42.530346 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 929/1830] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 12:24:42 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 12:24:42.533578 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24dfcf9a-2399-48dc-adc5-35fc67832b3a admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:24:42.633332 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24dfcf9a-2399-48dc-adc5-35fc67832b3a admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:24:42.640075 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c26ed9-c6c8-41ba-84c9-ec6c2362dfce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:24:42.640935 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c26ed9-c6c8-41ba-84c9-ec6c2362dfce admin admin] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:42.641370 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c26ed9-c6c8-41ba-84c9-ec6c2362dfce admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:42.641816 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c26ed9-c6c8-41ba-84c9-ec6c2362dfce admin admin] PATH_INFO: `/v3/users` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:42.642746 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c26ed9-c6c8-41ba-84c9-ec6c2362dfce admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:24:42.649242 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c26ed9-c6c8-41ba-84c9-ec6c2362dfce admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:24:42.708837 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 902/1831] 10.222.0.15 () {60 vars in 1144 bytes} [Thu Sep 15 12:24:42 2022] POST /identity/v3/users => generated 289 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 12:24:42.713133 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0170f479-93b8-4997-8d38-ae87e8c8282d service nova] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:24:42.740379 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0170f479-93b8-4997-8d38-ae87e8c8282d service nova] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:24:42.763285 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e22d77e-dac5-4d1c-842b-1bd537d28593 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:24:42.766135 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e22d77e-dac5-4d1c-842b-1bd537d28593 admin admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:42.766427 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e22d77e-dac5-4d1c-842b-1bd537d28593 admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:42.766708 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e22d77e-dac5-4d1c-842b-1bd537d28593 admin admin] PATH_INFO: `/v3/roles` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:42.767386 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e22d77e-dac5-4d1c-842b-1bd537d28593 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:24:42.777560 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e22d77e-dac5-4d1c-842b-1bd537d28593 admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:24:42.817812 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 930/1832] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 12:24:42 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 12:24:42.818138 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c26ed9-c6c8-41ba-84c9-ec6c2362dfce admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:24:42.830569 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c26ed9-c6c8-41ba-84c9-ec6c2362dfce admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:24:42.845816 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1527c6-c6ec-4213-b6d1-bc40d97bfd83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:24:42.846517 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1527c6-c6ec-4213-b6d1-bc40d97bfd83 admin admin] REQUEST_METHOD: `PUT` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:42.846794 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1527c6-c6ec-4213-b6d1-bc40d97bfd83 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:42.847158 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1527c6-c6ec-4213-b6d1-bc40d97bfd83 admin admin] PATH_INFO: `/v3/projects/e340eac4cac7499f8ba077adffb10d2f/users/d6e019b8141843ffa6a84141eb595cba/roles/9c769b5c82884766b2c510b1f0f1364c` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:42.894458 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1527c6-c6ec-4213-b6d1-bc40d97bfd83 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e340eac4cac7499f8ba077adffb10d2f, user_id=d6e019b8141843ffa6a84141eb595cba, role_id=9c769b5c82884766b2c510b1f0f1364c)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:24:42.907537 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1527c6-c6ec-4213-b6d1-bc40d97bfd83 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:24:42.936801 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 903/1833] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 15 12:24:42 2022] PUT /identity/v3/projects/e340eac4cac7499f8ba077adffb10d2f/users/d6e019b8141843ffa6a84141eb595cba/roles/9c769b5c82884766b2c510b1f0f1364c => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:24:42.939701 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e22d77e-dac5-4d1c-842b-1bd537d28593 admin admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:24:42.950950 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e22d77e-dac5-4d1c-842b-1bd537d28593 admin admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:24:43.081291 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9a6614-5255-4d6c-9eff-2db109287f2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:24:43.081914 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9a6614-5255-4d6c-9eff-2db109287f2e admin admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:43.082191 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9a6614-5255-4d6c-9eff-2db109287f2e admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:43.082462 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9a6614-5255-4d6c-9eff-2db109287f2e admin admin] PATH_INFO: `/v3/roles` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:43.083104 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b9a6614-5255-4d6c-9eff-2db109287f2e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:24:43.084707 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b9a6614-5255-4d6c-9eff-2db109287f2e admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:24:43.109886 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6cab35-50c2-408b-871e-a87b36c46fb5 None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:43.110171 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6cab35-50c2-408b-871e-a87b36c46fb5 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:43.110449 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6cab35-50c2-408b-871e-a87b36c46fb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:43.121833 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 931/1834] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 12:24:42 2022] GET /identity/v3/roles => generated 1420 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 12:24:43.126060 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9a6614-5255-4d6c-9eff-2db109287f2e admin admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:24:43.137211 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9a6614-5255-4d6c-9eff-2db109287f2e admin admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:24:43.142516 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45bf7a28-a6a1-475a-9684-f3ecbd0734b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:24:43.143185 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bf7a28-a6a1-475a-9684-f3ecbd0734b1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:43.143461 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bf7a28-a6a1-475a-9684-f3ecbd0734b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:43.143731 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bf7a28-a6a1-475a-9684-f3ecbd0734b1 admin admin] PATH_INFO: `/v3/projects/e340eac4cac7499f8ba077adffb10d2f/users/d6e019b8141843ffa6a84141eb595cba/roles/9c769b5c82884766b2c510b1f0f1364c` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:43.145662 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45bf7a28-a6a1-475a-9684-f3ecbd0734b1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e340eac4cac7499f8ba077adffb10d2f, user_id=d6e019b8141843ffa6a84141eb595cba, role_id=9c769b5c82884766b2c510b1f0f1364c)` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:24:43.148596 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45bf7a28-a6a1-475a-9684-f3ecbd0734b1 admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:24:43.162427 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-ad6cab35-50c2-408b-871e-a87b36c46fb5 None None] MFA Rules not processed for user `527f4dd6031546c785bc6360dacb60af`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:24:43.163833 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-ad6cab35-50c2-408b-871e-a87b36c46fb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:24:43.175659 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 932/1835] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 15 12:24:43 2022] PUT /identity/v3/projects/e340eac4cac7499f8ba077adffb10d2f/users/d6e019b8141843ffa6a84141eb595cba/roles/9c769b5c82884766b2c510b1f0f1364c => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 12:24:43.175659 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed7eb94-4b38-4779-9409-889ff40ec594 None None] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:43.175659 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed7eb94-4b38-4779-9409-889ff40ec594 None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:43.183398 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed7eb94-4b38-4779-9409-889ff40ec594 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:43.206456 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.auth.core [None req-9ed7eb94-4b38-4779-9409-889ff40ec594 None None] MFA Rules not processed for user `d6e019b8141843ffa6a84141eb595cba`. Rule list: `[]` (Enabled: `True`). {{(pid=81449) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:24:43.208580 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-9ed7eb94-4b38-4779-9409-889ff40ec594 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:24:43.216144 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:24:43.283704 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:24:43.309636 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 904/1836] 10.222.0.15 () {58 vars in 1009 bytes} [Thu Sep 15 12:24:43 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:24:43.325643 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6cab35-50c2-408b-871e-a87b36c46fb5 None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:24:43.327557 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-ad6cab35-50c2-408b-871e-a87b36c46fb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:24:43.360908 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 933/1837] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 12:24:43 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:24:43.371295 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed7eb94-4b38-4779-9409-889ff40ec594 None None] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:24:43.371592 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-9ed7eb94-4b38-4779-9409-889ff40ec594 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:24:43.483138 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6cab35-50c2-408b-871e-a87b36c46fb5 None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:24:43.488891 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998cf4bb-ec95-44bb-9929-0b32ed0b71f3 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa0d3dc5b7540d59fdd59bbf64a04b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '540a32ef88764e8c8d9201ae22f6ddaf', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:24:43.494308 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998cf4bb-ec95-44bb-9929-0b32ed0b71f3 service swift] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:43.494924 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998cf4bb-ec95-44bb-9929-0b32ed0b71f3 service swift] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:43.495348 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998cf4bb-ec95-44bb-9929-0b32ed0b71f3 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:43.499784 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed7eb94-4b38-4779-9409-889ff40ec594 None None] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:24:43.507419 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2983de1-47dd-4e59-9cce-7a3924b8dbaf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8191d3ccb874529b32f8d0026db629d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '540a32ef88764e8c8d9201ae22f6ddaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:24:43.508502 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2983de1-47dd-4e59-9cce-7a3924b8dbaf service cinder] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:43.508502 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2983de1-47dd-4e59-9cce-7a3924b8dbaf service cinder] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:43.508745 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2983de1-47dd-4e59-9cce-7a3924b8dbaf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:43.512015 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-998cf4bb-ec95-44bb-9929-0b32ed0b71f3 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:24:43.514100 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-998cf4bb-ec95-44bb-9929-0b32ed0b71f3 service swift] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:24:43.529428 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2983de1-47dd-4e59-9cce-7a3924b8dbaf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:24:43.532900 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 905/1838] 10.222.0.15 () {60 vars in 1384 bytes} [Thu Sep 15 12:24:43 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 208 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 15 12:24:43.533671 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2983de1-47dd-4e59-9cce-7a3924b8dbaf service cinder] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:24:43.568339 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 934/1839] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 12:24:43 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 198 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 15 12:24:43.722380 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0eb70c-65b0-449c-b75c-cd722ead3f4e None None] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:43.722380 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0eb70c-65b0-449c-b75c-cd722ead3f4e None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:43.722380 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0eb70c-65b0-449c-b75c-cd722ead3f4e None None] PATH_INFO: `/` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:43.724131 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 906/1840] 10.222.0.15 () {54 vars in 921 bytes} [Thu Sep 15 12:24:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:24:43.734144 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d56274f-e4f9-4bc7-8db7-9d8f2a25d29a None None] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:43.734359 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d56274f-e4f9-4bc7-8db7-9d8f2a25d29a None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:43.734609 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d56274f-e4f9-4bc7-8db7-9d8f2a25d29a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:43.745789 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0eb70c-65b0-449c-b75c-cd722ead3f4e None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:24:43.747697 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-0e0eb70c-65b0-449c-b75c-cd722ead3f4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:24:43.772136 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0eb70c-65b0-449c-b75c-cd722ead3f4e None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:24:43.776471 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef6bff3-2d0b-4b17-9e8e-aa47c55b0b29 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8191d3ccb874529b32f8d0026db629d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '540a32ef88764e8c8d9201ae22f6ddaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:24:43.778007 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef6bff3-2d0b-4b17-9e8e-aa47c55b0b29 service cinder] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:43.778308 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef6bff3-2d0b-4b17-9e8e-aa47c55b0b29 service cinder] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:43.778588 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef6bff3-2d0b-4b17-9e8e-aa47c55b0b29 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:43.782795 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.auth.core [None req-6d56274f-e4f9-4bc7-8db7-9d8f2a25d29a None None] MFA Rules not processed for user `51f64a058bd5440cb5268f228aa6e392`. Rule list: `[]` (Enabled: `True`). {{(pid=81449) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:24:43.786949 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-6d56274f-e4f9-4bc7-8db7-9d8f2a25d29a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:24:43.901788 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef6bff3-2d0b-4b17-9e8e-aa47c55b0b29 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:24:43.903535 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef6bff3-2d0b-4b17-9e8e-aa47c55b0b29 service cinder] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:24:43.935535 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 907/1841] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 12:24:43 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 12:24:44.000912 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 935/1842] 10.222.0.15 () {58 vars in 1018 bytes} [Thu Sep 15 12:24:43 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 268 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:24:44.030100 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef6bff3-2d0b-4b17-9e8e-aa47c55b0b29 service cinder] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:24:44.032529 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-8ef6bff3-2d0b-4b17-9e8e-aa47c55b0b29 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:24:44.059739 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef6bff3-2d0b-4b17-9e8e-aa47c55b0b29 service cinder] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:24:44.070206 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616a4750-075c-4017-98da-260d9f460104 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f64a058bd5440cb5268f228aa6e392', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '540a32ef88764e8c8d9201ae22f6ddaf', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:24:44.070811 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616a4750-075c-4017-98da-260d9f460104 service placement] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:44.071097 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616a4750-075c-4017-98da-260d9f460104 service placement] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:44.071406 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616a4750-075c-4017-98da-260d9f460104 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:44.118351 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616a4750-075c-4017-98da-260d9f460104 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:24:44.119962 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616a4750-075c-4017-98da-260d9f460104 service placement] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:24:44.161263 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 908/1843] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 12:24:44 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 12:24:47.310546 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7fda1f-e264-42f0-b2ac-901a037627a2 None None] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:47.311165 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7fda1f-e264-42f0-b2ac-901a037627a2 None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:47.311165 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7fda1f-e264-42f0-b2ac-901a037627a2 None None] PATH_INFO: `/` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:47.314603 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 936/1844] 10.222.0.15 () {54 vars in 921 bytes} [Thu Sep 15 12:24:47 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:24:47.357217 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832ebd95-7ca2-4b5e-aa8a-f1cf0b84c51d None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:47.358170 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832ebd95-7ca2-4b5e-aa8a-f1cf0b84c51d None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:47.358631 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832ebd95-7ca2-4b5e-aa8a-f1cf0b84c51d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:47.409168 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-832ebd95-7ca2-4b5e-aa8a-f1cf0b84c51d None None] MFA Rules not processed for user `51f64a058bd5440cb5268f228aa6e392`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:24:47.409168 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-832ebd95-7ca2-4b5e-aa8a-f1cf0b84c51d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:24:47.434512 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 909/1845] 10.222.0.15 () {58 vars in 1018 bytes} [Thu Sep 15 12:24:47 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (3 switches on core 0) Sep 15 12:24:47.522403 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03acdef5-bc37-40a0-9ab5-b9a1ec03c990 None None] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:47.522948 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03acdef5-bc37-40a0-9ab5-b9a1ec03c990 None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:47.523364 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03acdef5-bc37-40a0-9ab5-b9a1ec03c990 None None] PATH_INFO: `/` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:47.527169 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 937/1846] 10.222.0.15 () {54 vars in 961 bytes} [Thu Sep 15 12:24:47 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:24:47.535031 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9142d9e-82bb-46c7-8ebb-b301c6041aac None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:47.535643 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9142d9e-82bb-46c7-8ebb-b301c6041aac None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:47.536192 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9142d9e-82bb-46c7-8ebb-b301c6041aac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:47.567644 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03acdef5-bc37-40a0-9ab5-b9a1ec03c990 None None] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:24:47.570164 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-03acdef5-bc37-40a0-9ab5-b9a1ec03c990 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:24:47.630812 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-e9142d9e-82bb-46c7-8ebb-b301c6041aac None None] MFA Rules not processed for user `51f64a058bd5440cb5268f228aa6e392`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:24:47.632137 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-e9142d9e-82bb-46c7-8ebb-b301c6041aac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:24:47.667039 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 910/1847] 10.222.0.15 () {58 vars in 1058 bytes} [Thu Sep 15 12:24:47 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:24:47.694487 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9142d9e-82bb-46c7-8ebb-b301c6041aac None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:24:47.712972 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9142d9e-82bb-46c7-8ebb-b301c6041aac None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:24:47.736426 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a808ecc-30e5-40e4-bdb6-ea50de139945 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f64a058bd5440cb5268f228aa6e392', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '540a32ef88764e8c8d9201ae22f6ddaf', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:24:47.739812 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a808ecc-30e5-40e4-bdb6-ea50de139945 service placement] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:47.740488 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a808ecc-30e5-40e4-bdb6-ea50de139945 service placement] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:47.741084 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a808ecc-30e5-40e4-bdb6-ea50de139945 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:47.742281 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:24:47.777564 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a808ecc-30e5-40e4-bdb6-ea50de139945 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:24:47.777776 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03acdef5-bc37-40a0-9ab5-b9a1ec03c990 None None] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:24:47.781740 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a808ecc-30e5-40e4-bdb6-ea50de139945 service placement] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:24:47.783479 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b966f3-7fdf-41ab-b27f-515ef17e4291 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:24:47.784306 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b966f3-7fdf-41ab-b27f-515ef17e4291 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:47.784926 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b966f3-7fdf-41ab-b27f-515ef17e4291 admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:47.784926 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b966f3-7fdf-41ab-b27f-515ef17e4291 admin admin] PATH_INFO: `/v3/users/5c5f1c7f322d4ea8a7f62005f4296d64` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:47.786224 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b966f3-7fdf-41ab-b27f-515ef17e4291 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5c5f1c7f322d4ea8a7f62005f4296d64)` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:24:47.789498 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63b966f3-7fdf-41ab-b27f-515ef17e4291 admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:24:47.821051 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 911/1848] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 12:24:47 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 15 12:24:47.874301 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-63b966f3-7fdf-41ab-b27f-515ef17e4291 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5c5f1c7f322d4ea8a7f62005f4296d64'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:24:47.887084 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-63b966f3-7fdf-41ab-b27f-515ef17e4291 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5c5f1c7f322d4ea8a7f62005f4296d64'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:24:47.890016 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a808ecc-30e5-40e4-bdb6-ea50de139945 service placement] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:24:47.907237 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a808ecc-30e5-40e4-bdb6-ea50de139945 service placement] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:24:47.909094 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-63b966f3-7fdf-41ab-b27f-515ef17e4291 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5c5f1c7f322d4ea8a7f62005f4296d64'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:24:47.909969 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-63b966f3-7fdf-41ab-b27f-515ef17e4291 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5c5f1c7f322d4ea8a7f62005f4296d64'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:24:47.917583 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e98843-02e6-4a09-b59a-7a8579f7b2e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:24:47.918590 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e98843-02e6-4a09-b59a-7a8579f7b2e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:47.919101 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e98843-02e6-4a09-b59a-7a8579f7b2e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:47.919440 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e98843-02e6-4a09-b59a-7a8579f7b2e3 admin admin] PATH_INFO: `/v3/users/fca9eb3e04884525b788369196668d2b` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:47.921649 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e98843-02e6-4a09-b59a-7a8579f7b2e3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fca9eb3e04884525b788369196668d2b)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:24:47.923669 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e98843-02e6-4a09-b59a-7a8579f7b2e3 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:24:47.931424 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-63b966f3-7fdf-41ab-b27f-515ef17e4291 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5c5f1c7f322d4ea8a7f62005f4296d64'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:24:47.949606 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 938/1849] 10.222.0.15 () {58 vars in 1224 bytes} [Thu Sep 15 12:24:47 2022] DELETE /identity/v3/users/5c5f1c7f322d4ea8a7f62005f4296d64 => generated 0 bytes in 382 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 12:24:48.016189 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:24:48.034791 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-96e98843-02e6-4a09-b59a-7a8579f7b2e3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fca9eb3e04884525b788369196668d2b'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:24:48.055084 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-96e98843-02e6-4a09-b59a-7a8579f7b2e3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fca9eb3e04884525b788369196668d2b'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:24:48.070951 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-96e98843-02e6-4a09-b59a-7a8579f7b2e3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fca9eb3e04884525b788369196668d2b'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:24:48.073577 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-96e98843-02e6-4a09-b59a-7a8579f7b2e3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fca9eb3e04884525b788369196668d2b'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:24:48.094218 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-96e98843-02e6-4a09-b59a-7a8579f7b2e3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fca9eb3e04884525b788369196668d2b'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:24:48.119430 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 912/1850] 10.222.0.15 () {58 vars in 1224 bytes} [Thu Sep 15 12:24:47 2022] DELETE /identity/v3/users/fca9eb3e04884525b788369196668d2b => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 12:24:48.124421 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25326fd7-ba4d-44da-98fa-85d9651a4b18 None None] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:48.124720 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25326fd7-ba4d-44da-98fa-85d9651a4b18 None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:48.125024 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25326fd7-ba4d-44da-98fa-85d9651a4b18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:48.148385 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.auth.core [None req-25326fd7-ba4d-44da-98fa-85d9651a4b18 None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81449) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:24:48.151272 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-25326fd7-ba4d-44da-98fa-85d9651a4b18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:24:48.351039 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 939/1851] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 12:24:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:24:48.400518 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e98843-02e6-4a09-b59a-7a8579f7b2e3 admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:24:48.402810 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-96e98843-02e6-4a09-b59a-7a8579f7b2e3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:24:48.496813 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e98843-02e6-4a09-b59a-7a8579f7b2e3 admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:24:48.505699 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8258c7fe-ac44-4c6a-ab0a-f782a7f674db service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfbb4d6809064f9cb9e06626ed7d6814', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '540a32ef88764e8c8d9201ae22f6ddaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:24:48.507633 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8258c7fe-ac44-4c6a-ab0a-f782a7f674db service neutron] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:48.507914 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8258c7fe-ac44-4c6a-ab0a-f782a7f674db service neutron] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:48.509301 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8258c7fe-ac44-4c6a-ab0a-f782a7f674db service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:48.524661 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8258c7fe-ac44-4c6a-ab0a-f782a7f674db service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:24:48.527773 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8258c7fe-ac44-4c6a-ab0a-f782a7f674db service neutron] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:24:48.557081 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 913/1852] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 12:24:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 12:24:48.611235 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25326fd7-ba4d-44da-98fa-85d9651a4b18 None None] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:24:48.614532 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-25326fd7-ba4d-44da-98fa-85d9651a4b18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:24:48.665669 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25326fd7-ba4d-44da-98fa-85d9651a4b18 None None] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:24:48.672730 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b377896c-79b9-4229-9987-e4abf893dd6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:24:48.674269 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b377896c-79b9-4229-9987-e4abf893dd6c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:48.674738 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b377896c-79b9-4229-9987-e4abf893dd6c admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:48.675131 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b377896c-79b9-4229-9987-e4abf893dd6c admin admin] PATH_INFO: `/v3/projects/46235ae7cb404d7f8b3817acdca81a68` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:48.676459 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b377896c-79b9-4229-9987-e4abf893dd6c admin admin] RBAC: Authorizing `identity:delete_project(project_id=46235ae7cb404d7f8b3817acdca81a68)` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:24:48.679646 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b377896c-79b9-4229-9987-e4abf893dd6c admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:24:48.758514 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-b377896c-79b9-4229-9987-e4abf893dd6c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '46235ae7cb404d7f8b3817acdca81a68', 'request_id': 'req-b377896c-79b9-4229-9987-e4abf893dd6c'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:24:48.768975 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-b377896c-79b9-4229-9987-e4abf893dd6c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '46235ae7cb404d7f8b3817acdca81a68', 'request_id': 'req-b377896c-79b9-4229-9987-e4abf893dd6c'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:24:48.770568 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:24:48.778724 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-b377896c-79b9-4229-9987-e4abf893dd6c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '46235ae7cb404d7f8b3817acdca81a68', 'request_id': 'req-b377896c-79b9-4229-9987-e4abf893dd6c'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:24:48.797044 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-b377896c-79b9-4229-9987-e4abf893dd6c admin admin] The token cache is being invalidate because project 46235ae7cb404d7f8b3817acdca81a68 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81449) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 12:24:48.797465 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-b377896c-79b9-4229-9987-e4abf893dd6c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:24:48.798992 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-b377896c-79b9-4229-9987-e4abf893dd6c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:24:48.803736 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 940/1853] 10.222.0.15 () {58 vars in 1233 bytes} [Thu Sep 15 12:24:48 2022] DELETE /identity/v3/projects/46235ae7cb404d7f8b3817acdca81a68 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:24:48.806458 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8258c7fe-ac44-4c6a-ab0a-f782a7f674db service neutron] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:24:48.820529 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8258c7fe-ac44-4c6a-ab0a-f782a7f674db service neutron] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:24:48.962084 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1d7056-a2e8-45e2-8777-4fb6de72b63c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:24:48.962765 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1d7056-a2e8-45e2-8777-4fb6de72b63c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:48.963048 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1d7056-a2e8-45e2-8777-4fb6de72b63c admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:48.963317 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1d7056-a2e8-45e2-8777-4fb6de72b63c admin admin] PATH_INFO: `/v3/users/3976c4c8f9704a1db410926eb62fa31e` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:48.964909 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1d7056-a2e8-45e2-8777-4fb6de72b63c admin admin] RBAC: Authorizing `identity:delete_user(user_id=3976c4c8f9704a1db410926eb62fa31e)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:24:48.970085 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1d7056-a2e8-45e2-8777-4fb6de72b63c admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:24:49.065143 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-ff1d7056-a2e8-45e2-8777-4fb6de72b63c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3976c4c8f9704a1db410926eb62fa31e'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:24:49.075983 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:24:49.082962 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-ff1d7056-a2e8-45e2-8777-4fb6de72b63c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3976c4c8f9704a1db410926eb62fa31e'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:24:49.093488 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-ff1d7056-a2e8-45e2-8777-4fb6de72b63c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3976c4c8f9704a1db410926eb62fa31e'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:24:49.096777 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-ff1d7056-a2e8-45e2-8777-4fb6de72b63c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3976c4c8f9704a1db410926eb62fa31e'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:24:49.104486 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-ff1d7056-a2e8-45e2-8777-4fb6de72b63c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3976c4c8f9704a1db410926eb62fa31e'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:24:49.111187 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 914/1854] 10.222.0.15 () {58 vars in 1224 bytes} [Thu Sep 15 12:24:48 2022] DELETE /identity/v3/users/3976c4c8f9704a1db410926eb62fa31e => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:24:49.800139 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b377896c-79b9-4229-9987-e4abf893dd6c admin admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:24:49.803263 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-b377896c-79b9-4229-9987-e4abf893dd6c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:24:49.810114 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1d7056-a2e8-45e2-8777-4fb6de72b63c admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:24:49.811666 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-ff1d7056-a2e8-45e2-8777-4fb6de72b63c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:24:49.832623 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b377896c-79b9-4229-9987-e4abf893dd6c admin admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:24:49.975805 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4936c8c-65c2-47cc-a7dd-a9bd949f0378 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:24:49.976673 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4936c8c-65c2-47cc-a7dd-a9bd949f0378 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:49.977018 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4936c8c-65c2-47cc-a7dd-a9bd949f0378 admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:49.977954 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4936c8c-65c2-47cc-a7dd-a9bd949f0378 admin admin] PATH_INFO: `/v3/projects/927baf46d46845208f83d9b89e1eb6d3` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:49.978079 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1d7056-a2e8-45e2-8777-4fb6de72b63c admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:24:49.982383 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4936c8c-65c2-47cc-a7dd-a9bd949f0378 admin admin] RBAC: Authorizing `identity:delete_project(project_id=927baf46d46845208f83d9b89e1eb6d3)` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:24:49.984381 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4936c8c-65c2-47cc-a7dd-a9bd949f0378 admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:24:49.997084 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42be4d8d-e501-4888-99d6-1c65f6587d35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:24:49.997756 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42be4d8d-e501-4888-99d6-1c65f6587d35 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:49.998035 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42be4d8d-e501-4888-99d6-1c65f6587d35 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:49.998304 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42be4d8d-e501-4888-99d6-1c65f6587d35 admin admin] PATH_INFO: `/v3/projects/d018f19bc21a4471ba7577551ed60dfe` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:50.008032 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42be4d8d-e501-4888-99d6-1c65f6587d35 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d018f19bc21a4471ba7577551ed60dfe)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:24:50.008032 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42be4d8d-e501-4888-99d6-1c65f6587d35 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:24:50.085010 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-42be4d8d-e501-4888-99d6-1c65f6587d35 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd018f19bc21a4471ba7577551ed60dfe', 'request_id': 'req-42be4d8d-e501-4888-99d6-1c65f6587d35'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:24:50.100872 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-42be4d8d-e501-4888-99d6-1c65f6587d35 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd018f19bc21a4471ba7577551ed60dfe', 'request_id': 'req-42be4d8d-e501-4888-99d6-1c65f6587d35'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:24:50.101151 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-a4936c8c-65c2-47cc-a7dd-a9bd949f0378 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '927baf46d46845208f83d9b89e1eb6d3', 'request_id': 'req-a4936c8c-65c2-47cc-a7dd-a9bd949f0378'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:24:50.111493 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-a4936c8c-65c2-47cc-a7dd-a9bd949f0378 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '927baf46d46845208f83d9b89e1eb6d3', 'request_id': 'req-a4936c8c-65c2-47cc-a7dd-a9bd949f0378'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:24:50.122131 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-42be4d8d-e501-4888-99d6-1c65f6587d35 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd018f19bc21a4471ba7577551ed60dfe', 'request_id': 'req-42be4d8d-e501-4888-99d6-1c65f6587d35'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:24:50.122708 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-a4936c8c-65c2-47cc-a7dd-a9bd949f0378 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '927baf46d46845208f83d9b89e1eb6d3', 'request_id': 'req-a4936c8c-65c2-47cc-a7dd-a9bd949f0378'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:24:50.136360 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-42be4d8d-e501-4888-99d6-1c65f6587d35 admin admin] The token cache is being invalidate because project d018f19bc21a4471ba7577551ed60dfe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81448) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 12:24:50.137114 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-42be4d8d-e501-4888-99d6-1c65f6587d35 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:24:50.137645 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-42be4d8d-e501-4888-99d6-1c65f6587d35 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:24:50.140012 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 915/1855] 10.222.0.15 () {58 vars in 1233 bytes} [Thu Sep 15 12:24:49 2022] DELETE /identity/v3/projects/d018f19bc21a4471ba7577551ed60dfe => generated 0 bytes in 330 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 12:24:50.142234 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-a4936c8c-65c2-47cc-a7dd-a9bd949f0378 admin admin] The token cache is being invalidate because project 927baf46d46845208f83d9b89e1eb6d3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81449) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 12:24:50.143006 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-a4936c8c-65c2-47cc-a7dd-a9bd949f0378 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:24:50.146595 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70364f7c-ea0e-4d52-942f-223d05a631da None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:50.146966 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-a4936c8c-65c2-47cc-a7dd-a9bd949f0378 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:24:50.147079 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70364f7c-ea0e-4d52-942f-223d05a631da None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:50.147532 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70364f7c-ea0e-4d52-942f-223d05a631da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:50.151027 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 941/1856] 10.222.0.15 () {58 vars in 1233 bytes} [Thu Sep 15 12:24:49 2022] DELETE /identity/v3/projects/927baf46d46845208f83d9b89e1eb6d3 => generated 0 bytes in 351 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:24:50.156465 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4936c8c-65c2-47cc-a7dd-a9bd949f0378 admin admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:24:50.159400 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-a4936c8c-65c2-47cc-a7dd-a9bd949f0378 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:24:50.199465 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-70364f7c-ea0e-4d52-942f-223d05a631da None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:24:50.202682 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-70364f7c-ea0e-4d52-942f-223d05a631da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:24:50.307522 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 916/1857] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 12:24:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:24:50.309340 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4936c8c-65c2-47cc-a7dd-a9bd949f0378 admin admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:24:50.316517 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21ca41d-6d37-4ed6-8c94-fd848f854092 None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:50.316708 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21ca41d-6d37-4ed6-8c94-fd848f854092 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:50.316946 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21ca41d-6d37-4ed6-8c94-fd848f854092 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:50.321495 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8695cc3b-6551-489c-b9c6-2d8ffc14c664 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:24:50.322756 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8695cc3b-6551-489c-b9c6-2d8ffc14c664 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:50.323202 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8695cc3b-6551-489c-b9c6-2d8ffc14c664 admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:50.323715 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8695cc3b-6551-489c-b9c6-2d8ffc14c664 admin admin] PATH_INFO: `/v3/users/d6e019b8141843ffa6a84141eb595cba` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:50.325347 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8695cc3b-6551-489c-b9c6-2d8ffc14c664 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d6e019b8141843ffa6a84141eb595cba)` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:24:50.328688 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8695cc3b-6551-489c-b9c6-2d8ffc14c664 admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:24:50.332200 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=89,keystone:DELETE=9,keystone:INSERT=2 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:24:50.343297 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-b21ca41d-6d37-4ed6-8c94-fd848f854092 None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:24:50.346452 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-b21ca41d-6d37-4ed6-8c94-fd848f854092 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:24:50.352256 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:24:50.359165 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 917/1858] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 12:24:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 12:24:50.365970 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21ca41d-6d37-4ed6-8c94-fd848f854092 None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:24:50.382015 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21ca41d-6d37-4ed6-8c94-fd848f854092 None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:24:50.389508 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac48cc4a-c071-4616-bc5e-813d23779d16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:24:50.390133 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac48cc4a-c071-4616-bc5e-813d23779d16 admin admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:50.390411 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac48cc4a-c071-4616-bc5e-813d23779d16 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:50.390680 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac48cc4a-c071-4616-bc5e-813d23779d16 admin admin] PATH_INFO: `/v3/domains` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:50.391476 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac48cc4a-c071-4616-bc5e-813d23779d16 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:24:50.391783 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac48cc4a-c071-4616-bc5e-813d23779d16 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:24:50.393453 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac48cc4a-c071-4616-bc5e-813d23779d16 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:24:50.405130 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-8695cc3b-6551-489c-b9c6-2d8ffc14c664 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd6e019b8141843ffa6a84141eb595cba'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:24:50.407539 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 918/1859] 10.222.0.15 () {58 vars in 1153 bytes} [Thu Sep 15 12:24:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:24:50.410895 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac48cc4a-c071-4616-bc5e-813d23779d16 admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:24:50.415843 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-8695cc3b-6551-489c-b9c6-2d8ffc14c664 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd6e019b8141843ffa6a84141eb595cba'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:24:50.426671 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac48cc4a-c071-4616-bc5e-813d23779d16 admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:24:50.430966 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-8695cc3b-6551-489c-b9c6-2d8ffc14c664 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd6e019b8141843ffa6a84141eb595cba'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:24:50.431987 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-8695cc3b-6551-489c-b9c6-2d8ffc14c664 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd6e019b8141843ffa6a84141eb595cba'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:24:50.433084 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba92cc02-6ee9-4cfd-90bc-c01d153a9314 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:24:50.433894 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba92cc02-6ee9-4cfd-90bc-c01d153a9314 admin admin] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:50.436993 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba92cc02-6ee9-4cfd-90bc-c01d153a9314 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:50.437310 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba92cc02-6ee9-4cfd-90bc-c01d153a9314 admin admin] PATH_INFO: `/v3/projects` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:50.437982 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba92cc02-6ee9-4cfd-90bc-c01d153a9314 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:24:50.440817 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba92cc02-6ee9-4cfd-90bc-c01d153a9314 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:24:50.443132 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-8695cc3b-6551-489c-b9c6-2d8ffc14c664 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd6e019b8141843ffa6a84141eb595cba'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:24:50.455471 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 942/1860] 10.222.0.15 () {58 vars in 1224 bytes} [Thu Sep 15 12:24:50 2022] DELETE /identity/v3/users/d6e019b8141843ffa6a84141eb595cba => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:24:50.501462 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 919/1861] 10.222.0.15 () {60 vars in 1153 bytes} [Thu Sep 15 12:24:50 2022] POST /identity/v3/projects => generated 369 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 15 12:24:50.506322 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8695cc3b-6551-489c-b9c6-2d8ffc14c664 admin admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:24:50.512537 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-8695cc3b-6551-489c-b9c6-2d8ffc14c664 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:24:50.613063 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8695cc3b-6551-489c-b9c6-2d8ffc14c664 admin admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:24:50.687533 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff0698d-4468-480c-9df5-0d8ff4f5f282 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:24:50.687533 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff0698d-4468-480c-9df5-0d8ff4f5f282 admin admin] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:50.687888 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff0698d-4468-480c-9df5-0d8ff4f5f282 admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:50.688224 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff0698d-4468-480c-9df5-0d8ff4f5f282 admin admin] PATH_INFO: `/v3/users` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:50.689385 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff0698d-4468-480c-9df5-0d8ff4f5f282 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:24:50.691142 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff0698d-4468-480c-9df5-0d8ff4f5f282 admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:24:50.716640 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 943/1862] 10.222.0.15 () {60 vars in 1144 bytes} [Thu Sep 15 12:24:50 2022] POST /identity/v3/users => generated 285 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 12:24:50.721260 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba92cc02-6ee9-4cfd-90bc-c01d153a9314 admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:24:50.731872 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba92cc02-6ee9-4cfd-90bc-c01d153a9314 admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:24:50.736929 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f598069e-66be-4c55-b6e3-7cfc744e2845 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:24:50.737567 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f598069e-66be-4c55-b6e3-7cfc744e2845 admin admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:50.737845 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f598069e-66be-4c55-b6e3-7cfc744e2845 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:50.738113 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f598069e-66be-4c55-b6e3-7cfc744e2845 admin admin] PATH_INFO: `/v3/roles` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:50.738749 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f598069e-66be-4c55-b6e3-7cfc744e2845 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:24:50.740336 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f598069e-66be-4c55-b6e3-7cfc744e2845 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:24:50.754627 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 920/1863] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 12:24:50 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 12:24:50.755773 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff0698d-4468-480c-9df5-0d8ff4f5f282 admin admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:24:50.774153 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff0698d-4468-480c-9df5-0d8ff4f5f282 admin admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:24:50.779179 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6de7d4-e3e8-42d0-a37e-0392dfe5446d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:24:50.779799 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6de7d4-e3e8-42d0-a37e-0392dfe5446d admin admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:50.780125 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6de7d4-e3e8-42d0-a37e-0392dfe5446d admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:50.780651 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6de7d4-e3e8-42d0-a37e-0392dfe5446d admin admin] PATH_INFO: `/v3/roles` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:50.781401 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f6de7d4-e3e8-42d0-a37e-0392dfe5446d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:24:50.783780 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f6de7d4-e3e8-42d0-a37e-0392dfe5446d admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:24:50.802746 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 944/1864] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 12:24:50 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 15 12:24:50.807157 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f598069e-66be-4c55-b6e3-7cfc744e2845 admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:24:50.834160 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f598069e-66be-4c55-b6e3-7cfc744e2845 admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:24:50.841811 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a54c1d7-4714-462d-88d5-94c66490aec1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:24:50.842529 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a54c1d7-4714-462d-88d5-94c66490aec1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:50.842781 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a54c1d7-4714-462d-88d5-94c66490aec1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:50.843053 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a54c1d7-4714-462d-88d5-94c66490aec1 admin admin] PATH_INFO: `/v3/projects/e785b10079b84944b7143167560e8385/users/205b2ef591c641838ca51affce1f4c0c/roles/0679d6628a25438d90dc4bdc0401eda2` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:50.860361 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a54c1d7-4714-462d-88d5-94c66490aec1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e785b10079b84944b7143167560e8385, user_id=205b2ef591c641838ca51affce1f4c0c, role_id=0679d6628a25438d90dc4bdc0401eda2)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:24:50.864303 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a54c1d7-4714-462d-88d5-94c66490aec1 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:24:50.893157 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 921/1865] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 15 12:24:50 2022] PUT /identity/v3/projects/e785b10079b84944b7143167560e8385/users/205b2ef591c641838ca51affce1f4c0c/roles/0679d6628a25438d90dc4bdc0401eda2 => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:24:50.903527 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5198a86-dfa3-4118-a1dc-c1dd77d39a18 None None] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:50.904189 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5198a86-dfa3-4118-a1dc-c1dd77d39a18 None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:50.904189 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5198a86-dfa3-4118-a1dc-c1dd77d39a18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:50.934154 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.auth.core [None req-d5198a86-dfa3-4118-a1dc-c1dd77d39a18 None None] MFA Rules not processed for user `205b2ef591c641838ca51affce1f4c0c`. Rule list: `[]` (Enabled: `True`). {{(pid=81449) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:24:50.935369 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-d5198a86-dfa3-4118-a1dc-c1dd77d39a18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:24:50.943532 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:DELETE=9,keystone:SELECT=87,keystone:INSERT=4 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:24:51.019454 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a54c1d7-4714-462d-88d5-94c66490aec1 admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:24:51.026528 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-7a54c1d7-4714-462d-88d5-94c66490aec1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:24:51.032838 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 945/1866] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 12:24:50 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:24:51.037484 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5198a86-dfa3-4118-a1dc-c1dd77d39a18 None None] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:24:51.166394 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5198a86-dfa3-4118-a1dc-c1dd77d39a18 None None] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:24:51.171585 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720fb01f-186c-43c1-897c-f50bb445fe8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:24:51.172385 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720fb01f-186c-43c1-897c-f50bb445fe8c admin admin] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:51.172734 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720fb01f-186c-43c1-897c-f50bb445fe8c admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:51.173297 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720fb01f-186c-43c1-897c-f50bb445fe8c admin admin] PATH_INFO: `/v3/projects` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:51.174116 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-720fb01f-186c-43c1-897c-f50bb445fe8c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:24:51.176221 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-720fb01f-186c-43c1-897c-f50bb445fe8c admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:24:51.188193 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a54c1d7-4714-462d-88d5-94c66490aec1 admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:24:51.198126 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744f4cd0-4163-4b6e-9f75-289776ab5d6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:24:51.198862 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744f4cd0-4163-4b6e-9f75-289776ab5d6a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:51.199318 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744f4cd0-4163-4b6e-9f75-289776ab5d6a admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:51.199778 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744f4cd0-4163-4b6e-9f75-289776ab5d6a admin admin] PATH_INFO: `/v3/projects/e340eac4cac7499f8ba077adffb10d2f` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:51.201501 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744f4cd0-4163-4b6e-9f75-289776ab5d6a admin admin] RBAC: Authorizing `identity:delete_project(project_id=e340eac4cac7499f8ba077adffb10d2f)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:24:51.213028 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744f4cd0-4163-4b6e-9f75-289776ab5d6a admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:24:51.242876 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 946/1867] 10.222.0.15 () {60 vars in 1153 bytes} [Thu Sep 15 12:24:51 2022] POST /identity/v3/projects => generated 367 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 12:24:51.248733 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720fb01f-186c-43c1-897c-f50bb445fe8c admin admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:24:51.319102 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-744f4cd0-4163-4b6e-9f75-289776ab5d6a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e340eac4cac7499f8ba077adffb10d2f', 'request_id': 'req-744f4cd0-4163-4b6e-9f75-289776ab5d6a'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:24:51.339492 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-744f4cd0-4163-4b6e-9f75-289776ab5d6a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e340eac4cac7499f8ba077adffb10d2f', 'request_id': 'req-744f4cd0-4163-4b6e-9f75-289776ab5d6a'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:24:51.353559 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-744f4cd0-4163-4b6e-9f75-289776ab5d6a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e340eac4cac7499f8ba077adffb10d2f', 'request_id': 'req-744f4cd0-4163-4b6e-9f75-289776ab5d6a'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:24:51.375379 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-744f4cd0-4163-4b6e-9f75-289776ab5d6a admin admin] The token cache is being invalidate because project e340eac4cac7499f8ba077adffb10d2f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81448) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 12:24:51.375845 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-744f4cd0-4163-4b6e-9f75-289776ab5d6a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:24:51.376666 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-744f4cd0-4163-4b6e-9f75-289776ab5d6a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:24:51.382699 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 922/1868] 10.222.0.15 () {58 vars in 1233 bytes} [Thu Sep 15 12:24:51 2022] DELETE /identity/v3/projects/e340eac4cac7499f8ba077adffb10d2f => generated 0 bytes in 360 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:24:51.386158 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc168df-0e65-4204-860f-edea8dea50ae None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:51.387221 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc168df-0e65-4204-860f-edea8dea50ae None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:51.387682 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc168df-0e65-4204-860f-edea8dea50ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:51.432191 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720fb01f-186c-43c1-897c-f50bb445fe8c admin admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:24:51.447154 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-7cc168df-0e65-4204-860f-edea8dea50ae None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:24:51.452285 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-7cc168df-0e65-4204-860f-edea8dea50ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:24:51.502214 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:24:51.543431 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:24:51.576464 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b9d11c-7e18-4b5c-87e1-733095ee88dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:24:51.577339 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b9d11c-7e18-4b5c-87e1-733095ee88dc admin admin] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:51.577783 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b9d11c-7e18-4b5c-87e1-733095ee88dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:51.578201 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b9d11c-7e18-4b5c-87e1-733095ee88dc admin admin] PATH_INFO: `/v3/users` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:51.578995 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b9d11c-7e18-4b5c-87e1-733095ee88dc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:24:51.581110 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b9d11c-7e18-4b5c-87e1-733095ee88dc admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:24:51.590632 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 923/1869] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 12:24:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:24:51.599308 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b1571b-8f77-4ba1-8c98-3a249d5374e2 None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:51.599693 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b1571b-8f77-4ba1-8c98-3a249d5374e2 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:51.599947 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b1571b-8f77-4ba1-8c98-3a249d5374e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:51.631273 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-c4b1571b-8f77-4ba1-8c98-3a249d5374e2 None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:24:51.632727 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-c4b1571b-8f77-4ba1-8c98-3a249d5374e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:24:51.643069 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 947/1870] 10.222.0.15 () {60 vars in 1144 bytes} [Thu Sep 15 12:24:51 2022] POST /identity/v3/users => generated 290 bytes in 398 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 12:24:51.646675 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 924/1871] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 12:24:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:24:51.647873 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b1571b-8f77-4ba1-8c98-3a249d5374e2 None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:24:51.648144 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b9d11c-7e18-4b5c-87e1-733095ee88dc admin admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:24:51.653714 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-24b9d11c-7e18-4b5c-87e1-733095ee88dc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:24:51.663045 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b1571b-8f77-4ba1-8c98-3a249d5374e2 None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:24:51.684602 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b9d11c-7e18-4b5c-87e1-733095ee88dc admin admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:24:51.687215 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e70e069-adb4-474d-baa7-0d1d7fbe4283 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:24:51.688128 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e70e069-adb4-474d-baa7-0d1d7fbe4283 admin admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:51.688622 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e70e069-adb4-474d-baa7-0d1d7fbe4283 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:51.689639 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e70e069-adb4-474d-baa7-0d1d7fbe4283 admin admin] PATH_INFO: `/v3/domains` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:51.690861 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e70e069-adb4-474d-baa7-0d1d7fbe4283 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:24:51.691396 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e70e069-adb4-474d-baa7-0d1d7fbe4283 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:24:51.692833 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0ee56b-9181-4f93-ac00-a20a4d58060e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:24:51.693469 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0ee56b-9181-4f93-ac00-a20a4d58060e admin admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:51.693752 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0ee56b-9181-4f93-ac00-a20a4d58060e admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:51.694024 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0ee56b-9181-4f93-ac00-a20a4d58060e admin admin] PATH_INFO: `/v3/roles` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:51.694670 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0ee56b-9181-4f93-ac00-a20a4d58060e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:24:51.695322 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e70e069-adb4-474d-baa7-0d1d7fbe4283 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:24:51.696817 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0ee56b-9181-4f93-ac00-a20a4d58060e admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:24:51.730376 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 948/1872] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 12:24:51 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 15 12:24:51.744648 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0ee56b-9181-4f93-ac00-a20a4d58060e admin admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:24:51.746801 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 925/1873] 10.222.0.15 () {58 vars in 1153 bytes} [Thu Sep 15 12:24:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:24:51.747906 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e70e069-adb4-474d-baa7-0d1d7fbe4283 admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:24:51.762954 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0ee56b-9181-4f93-ac00-a20a4d58060e admin admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:24:51.768000 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c702d4-caed-4e5b-96b0-4a3935704679 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:24:51.768693 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c702d4-caed-4e5b-96b0-4a3935704679 admin admin] REQUEST_METHOD: `PUT` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:51.769053 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c702d4-caed-4e5b-96b0-4a3935704679 admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:51.769266 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c702d4-caed-4e5b-96b0-4a3935704679 admin admin] PATH_INFO: `/v3/projects/5bd93e804d234514829f2c01a1541151/users/bc577218c6ca4d0aa7548023ac2719f9/roles/9c769b5c82884766b2c510b1f0f1364c` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:51.771210 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e70e069-adb4-474d-baa7-0d1d7fbe4283 admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:24:51.786415 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362bb8a8-3ac7-4ae3-871c-c664a0a9af91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:24:51.787396 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362bb8a8-3ac7-4ae3-871c-c664a0a9af91 admin admin] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:51.788004 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362bb8a8-3ac7-4ae3-871c-c664a0a9af91 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:51.788452 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362bb8a8-3ac7-4ae3-871c-c664a0a9af91 admin admin] PATH_INFO: `/v3/projects` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:51.789945 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-362bb8a8-3ac7-4ae3-871c-c664a0a9af91 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:24:51.793124 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-362bb8a8-3ac7-4ae3-871c-c664a0a9af91 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:24:51.826557 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c702d4-caed-4e5b-96b0-4a3935704679 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5bd93e804d234514829f2c01a1541151, user_id=bc577218c6ca4d0aa7548023ac2719f9, role_id=9c769b5c82884766b2c510b1f0f1364c)` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:24:51.830720 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c702d4-caed-4e5b-96b0-4a3935704679 admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:24:51.868638 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 926/1874] 10.222.0.15 () {60 vars in 1153 bytes} [Thu Sep 15 12:24:51 2022] POST /identity/v3/projects => generated 381 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 12:24:51.872678 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362bb8a8-3ac7-4ae3-871c-c664a0a9af91 admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:24:51.875291 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 949/1875] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 15 12:24:51 2022] PUT /identity/v3/projects/5bd93e804d234514829f2c01a1541151/users/bc577218c6ca4d0aa7548023ac2719f9/roles/9c769b5c82884766b2c510b1f0f1364c => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:24:51.879142 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c702d4-caed-4e5b-96b0-4a3935704679 admin admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:24:52.029411 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c702d4-caed-4e5b-96b0-4a3935704679 admin admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:24:52.036332 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6255dc-a2e2-486b-9bdc-ccb173193f64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:24:52.037915 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6255dc-a2e2-486b-9bdc-ccb173193f64 admin admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:52.038614 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6255dc-a2e2-486b-9bdc-ccb173193f64 admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:52.039135 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6255dc-a2e2-486b-9bdc-ccb173193f64 admin admin] PATH_INFO: `/v3/roles` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:52.041464 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6255dc-a2e2-486b-9bdc-ccb173193f64 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:24:52.042485 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6255dc-a2e2-486b-9bdc-ccb173193f64 admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:24:52.049282 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362bb8a8-3ac7-4ae3-871c-c664a0a9af91 admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:24:52.061267 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829f7b26-85d1-4522-b627-c63ad7ab7bb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:24:52.061521 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 950/1876] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 12:24:51 2022] GET /identity/v3/roles => generated 1420 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 12:24:52.061651 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829f7b26-85d1-4522-b627-c63ad7ab7bb6 admin admin] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:52.061651 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829f7b26-85d1-4522-b627-c63ad7ab7bb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:52.061651 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829f7b26-85d1-4522-b627-c63ad7ab7bb6 admin admin] PATH_INFO: `/v3/users` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:52.061651 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-829f7b26-85d1-4522-b627-c63ad7ab7bb6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:24:52.062033 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-829f7b26-85d1-4522-b627-c63ad7ab7bb6 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:24:52.065228 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6255dc-a2e2-486b-9bdc-ccb173193f64 admin admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:24:52.095916 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 927/1877] 10.222.0.15 () {60 vars in 1144 bytes} [Thu Sep 15 12:24:51 2022] POST /identity/v3/users => generated 291 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 12:24:52.096138 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6255dc-a2e2-486b-9bdc-ccb173193f64 admin admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:24:52.098264 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829f7b26-85d1-4522-b627-c63ad7ab7bb6 admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:24:52.109673 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a05c322-2573-4af1-9dc1-31302096c125 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:24:52.110468 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a05c322-2573-4af1-9dc1-31302096c125 admin admin] REQUEST_METHOD: `PUT` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:52.111177 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829f7b26-85d1-4522-b627-c63ad7ab7bb6 admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:24:52.111580 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a05c322-2573-4af1-9dc1-31302096c125 admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:52.113060 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a05c322-2573-4af1-9dc1-31302096c125 admin admin] PATH_INFO: `/v3/projects/5bd93e804d234514829f2c01a1541151/users/bc577218c6ca4d0aa7548023ac2719f9/roles/9c769b5c82884766b2c510b1f0f1364c` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:52.117440 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b1d481-1dfd-4305-bfcb-146e5befcd15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:24:52.119553 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b1d481-1dfd-4305-bfcb-146e5befcd15 admin admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:52.119950 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b1d481-1dfd-4305-bfcb-146e5befcd15 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:52.120203 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a05c322-2573-4af1-9dc1-31302096c125 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5bd93e804d234514829f2c01a1541151, user_id=bc577218c6ca4d0aa7548023ac2719f9, role_id=9c769b5c82884766b2c510b1f0f1364c)` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:24:52.120412 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b1d481-1dfd-4305-bfcb-146e5befcd15 admin admin] PATH_INFO: `/v3/roles` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:52.121195 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b1d481-1dfd-4305-bfcb-146e5befcd15 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:24:52.122911 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b1d481-1dfd-4305-bfcb-146e5befcd15 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:24:52.123041 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a05c322-2573-4af1-9dc1-31302096c125 admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:24:52.140195 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 928/1878] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 12:24:52 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 12:24:52.141470 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b1d481-1dfd-4305-bfcb-146e5befcd15 admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:24:52.158888 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 951/1879] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 15 12:24:52 2022] PUT /identity/v3/projects/5bd93e804d234514829f2c01a1541151/users/bc577218c6ca4d0aa7548023ac2719f9/roles/9c769b5c82884766b2c510b1f0f1364c => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:24:52.168119 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b1d481-1dfd-4305-bfcb-146e5befcd15 admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:24:52.169058 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d561970-7f7f-4427-be18-7d19b78da1cf None None] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:52.169349 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d561970-7f7f-4427-be18-7d19b78da1cf None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:52.169628 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d561970-7f7f-4427-be18-7d19b78da1cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:52.232548 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.auth.core [None req-2d561970-7f7f-4427-be18-7d19b78da1cf None None] MFA Rules not processed for user `bc577218c6ca4d0aa7548023ac2719f9`. Rule list: `[]` (Enabled: `True`). {{(pid=81449) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:24:52.234045 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-2d561970-7f7f-4427-be18-7d19b78da1cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:24:52.255755 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:24:52.268844 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee59e87-38e7-40fd-8420-48e65db8df52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:24:52.269663 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee59e87-38e7-40fd-8420-48e65db8df52 admin admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:52.269949 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee59e87-38e7-40fd-8420-48e65db8df52 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:52.270730 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee59e87-38e7-40fd-8420-48e65db8df52 admin admin] PATH_INFO: `/v3/roles` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:52.271246 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee59e87-38e7-40fd-8420-48e65db8df52 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:24:52.272948 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee59e87-38e7-40fd-8420-48e65db8df52 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:24:52.288324 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 929/1880] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 12:24:52 2022] GET /identity/v3/roles => generated 1420 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 12:24:52.291828 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee59e87-38e7-40fd-8420-48e65db8df52 admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:24:52.307370 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee59e87-38e7-40fd-8420-48e65db8df52 admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:24:52.314124 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d85985-c650-4a98-9d1d-a005fcd041b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:24:52.315057 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d85985-c650-4a98-9d1d-a005fcd041b1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:52.315571 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d85985-c650-4a98-9d1d-a005fcd041b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:52.316186 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d85985-c650-4a98-9d1d-a005fcd041b1 admin admin] PATH_INFO: `/v3/projects/4696b497d7c54d049e79e5cfe2d000c3/users/19c29ddef4f74d05bd075846d76e0393/roles/0679d6628a25438d90dc4bdc0401eda2` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:52.333930 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d85985-c650-4a98-9d1d-a005fcd041b1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4696b497d7c54d049e79e5cfe2d000c3, user_id=19c29ddef4f74d05bd075846d76e0393, role_id=0679d6628a25438d90dc4bdc0401eda2)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:24:52.336616 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d85985-c650-4a98-9d1d-a005fcd041b1 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:24:52.352475 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 930/1881] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 15 12:24:52 2022] PUT /identity/v3/projects/4696b497d7c54d049e79e5cfe2d000c3/users/19c29ddef4f74d05bd075846d76e0393/roles/0679d6628a25438d90dc4bdc0401eda2 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 12:24:52.362643 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6413f6c8-717b-4441-a27e-0c9054a00581 None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:52.363094 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 952/1882] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 12:24:52 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:24:52.365067 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6413f6c8-717b-4441-a27e-0c9054a00581 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:52.365554 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6413f6c8-717b-4441-a27e-0c9054a00581 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:52.393748 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d561970-7f7f-4427-be18-7d19b78da1cf None None] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:24:52.395597 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-2d561970-7f7f-4427-be18-7d19b78da1cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:24:52.403934 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:24:52.405128 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-6413f6c8-717b-4441-a27e-0c9054a00581 None None] MFA Rules not processed for user `19c29ddef4f74d05bd075846d76e0393`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:24:52.407228 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-6413f6c8-717b-4441-a27e-0c9054a00581 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:24:52.463330 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d561970-7f7f-4427-be18-7d19b78da1cf None None] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:24:52.497176 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 931/1883] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 12:24:52 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 12:24:52.501096 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6413f6c8-717b-4441-a27e-0c9054a00581 None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:24:52.531836 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c97735e-49e8-4ab1-adc2-4a2a33f2dbac service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8191d3ccb874529b32f8d0026db629d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '540a32ef88764e8c8d9201ae22f6ddaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:24:52.532862 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c97735e-49e8-4ab1-adc2-4a2a33f2dbac service cinder] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:52.533158 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c97735e-49e8-4ab1-adc2-4a2a33f2dbac service cinder] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:52.533437 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c97735e-49e8-4ab1-adc2-4a2a33f2dbac service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:52.590587 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6413f6c8-717b-4441-a27e-0c9054a00581 None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:24:52.597871 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972bfced-7d08-409a-a803-051f1034fd1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:24:52.598878 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972bfced-7d08-409a-a803-051f1034fd1c admin admin] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:52.599229 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972bfced-7d08-409a-a803-051f1034fd1c admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:52.599582 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972bfced-7d08-409a-a803-051f1034fd1c admin admin] PATH_INFO: `/v3/projects` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:52.600509 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972bfced-7d08-409a-a803-051f1034fd1c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:24:52.603444 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972bfced-7d08-409a-a803-051f1034fd1c admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:24:52.611516 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c97735e-49e8-4ab1-adc2-4a2a33f2dbac service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:24:52.613012 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c97735e-49e8-4ab1-adc2-4a2a33f2dbac service cinder] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:24:52.625053 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 953/1884] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 12:24:52 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 232 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 12:24:52.637161 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 932/1885] 10.222.0.15 () {60 vars in 1153 bytes} [Thu Sep 15 12:24:52 2022] POST /identity/v3/projects => generated 379 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 15 12:24:52.641067 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c97735e-49e8-4ab1-adc2-4a2a33f2dbac service cinder] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:24:52.655734 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c97735e-49e8-4ab1-adc2-4a2a33f2dbac service cinder] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:24:52.729820 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771f72ab-0184-4f1a-b81e-e899bfdf4d95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:24:52.730572 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771f72ab-0184-4f1a-b81e-e899bfdf4d95 admin admin] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:52.730859 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771f72ab-0184-4f1a-b81e-e899bfdf4d95 admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:52.731125 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771f72ab-0184-4f1a-b81e-e899bfdf4d95 admin admin] PATH_INFO: `/v3/users` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:52.731920 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771f72ab-0184-4f1a-b81e-e899bfdf4d95 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:24:52.733718 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771f72ab-0184-4f1a-b81e-e899bfdf4d95 admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:24:52.754552 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:24:52.761498 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 954/1886] 10.222.0.15 () {60 vars in 1144 bytes} [Thu Sep 15 12:24:52 2022] POST /identity/v3/users => generated 296 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 15 12:24:52.765570 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972bfced-7d08-409a-a803-051f1034fd1c admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:24:52.776739 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972bfced-7d08-409a-a803-051f1034fd1c admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:24:52.785279 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f49f819-d659-4d2b-8a43-f66e60be496f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:24:52.786435 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f49f819-d659-4d2b-8a43-f66e60be496f admin admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:52.786862 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f49f819-d659-4d2b-8a43-f66e60be496f admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:52.787300 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f49f819-d659-4d2b-8a43-f66e60be496f admin admin] PATH_INFO: `/v3/roles` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:52.788379 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f49f819-d659-4d2b-8a43-f66e60be496f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:24:52.791161 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f49f819-d659-4d2b-8a43-f66e60be496f admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:24:52.804186 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 933/1887] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 12:24:52 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 12:24:52.809040 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771f72ab-0184-4f1a-b81e-e899bfdf4d95 admin admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:24:52.819695 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771f72ab-0184-4f1a-b81e-e899bfdf4d95 admin admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:24:52.825421 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04ea003-3565-4fc6-8a08-79bdb458b461 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:24:52.826306 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04ea003-3565-4fc6-8a08-79bdb458b461 admin admin] REQUEST_METHOD: `PUT` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:52.826982 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04ea003-3565-4fc6-8a08-79bdb458b461 admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:52.826982 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04ea003-3565-4fc6-8a08-79bdb458b461 admin admin] PATH_INFO: `/v3/projects/ef4f5543f5634d1d88240bb77c95fcfc/users/2fbea441705a4c4092d258fd5bf71cca/roles/9c769b5c82884766b2c510b1f0f1364c` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:52.845715 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04ea003-3565-4fc6-8a08-79bdb458b461 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ef4f5543f5634d1d88240bb77c95fcfc, user_id=2fbea441705a4c4092d258fd5bf71cca, role_id=9c769b5c82884766b2c510b1f0f1364c)` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:24:52.849207 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04ea003-3565-4fc6-8a08-79bdb458b461 admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:24:52.875551 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 955/1888] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 15 12:24:52 2022] PUT /identity/v3/projects/ef4f5543f5634d1d88240bb77c95fcfc/users/2fbea441705a4c4092d258fd5bf71cca/roles/9c769b5c82884766b2c510b1f0f1364c => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:24:52.879596 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f49f819-d659-4d2b-8a43-f66e60be496f admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:24:52.895345 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f49f819-d659-4d2b-8a43-f66e60be496f admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:24:52.948240 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:24:53.027979 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e15e30-c3fd-4b1b-994e-f5dc8efa083c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:24:53.028615 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e15e30-c3fd-4b1b-994e-f5dc8efa083c admin admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:53.028916 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e15e30-c3fd-4b1b-994e-f5dc8efa083c admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:53.029190 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e15e30-c3fd-4b1b-994e-f5dc8efa083c admin admin] PATH_INFO: `/v3/roles` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:53.029833 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e15e30-c3fd-4b1b-994e-f5dc8efa083c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:24:53.031494 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6e15e30-c3fd-4b1b-994e-f5dc8efa083c admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:24:53.050970 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 934/1889] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 12:24:52 2022] GET /identity/v3/roles => generated 1420 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 12:24:53.054974 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04ea003-3565-4fc6-8a08-79bdb458b461 admin admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:24:53.074729 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04ea003-3565-4fc6-8a08-79bdb458b461 admin admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:24:53.103706 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4443db25-20d3-48e5-9030-3ec70f401ca1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:24:53.115422 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4443db25-20d3-48e5-9030-3ec70f401ca1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:53.116069 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4443db25-20d3-48e5-9030-3ec70f401ca1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:53.124135 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4443db25-20d3-48e5-9030-3ec70f401ca1 admin admin] PATH_INFO: `/v3/projects/ef4f5543f5634d1d88240bb77c95fcfc/users/2fbea441705a4c4092d258fd5bf71cca/roles/9c769b5c82884766b2c510b1f0f1364c` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:53.124135 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4443db25-20d3-48e5-9030-3ec70f401ca1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ef4f5543f5634d1d88240bb77c95fcfc, user_id=2fbea441705a4c4092d258fd5bf71cca, role_id=9c769b5c82884766b2c510b1f0f1364c)` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:24:53.129753 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4443db25-20d3-48e5-9030-3ec70f401ca1 admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:24:53.155237 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 956/1890] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 15 12:24:53 2022] PUT /identity/v3/projects/ef4f5543f5634d1d88240bb77c95fcfc/users/2fbea441705a4c4092d258fd5bf71cca/roles/9c769b5c82884766b2c510b1f0f1364c => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:24:53.166264 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb0abe4-aea2-4a28-ad4b-236cbb1931e5 None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:53.166549 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb0abe4-aea2-4a28-ad4b-236cbb1931e5 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:53.166821 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb0abe4-aea2-4a28-ad4b-236cbb1931e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:53.272336 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-6eb0abe4-aea2-4a28-ad4b-236cbb1931e5 None None] MFA Rules not processed for user `2fbea441705a4c4092d258fd5bf71cca`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:24:53.273739 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-6eb0abe4-aea2-4a28-ad4b-236cbb1931e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:24:53.460805 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 935/1891] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 12:24:53 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 295 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:24:53.485666 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4443db25-20d3-48e5-9030-3ec70f401ca1 admin admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:24:53.488532 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-4443db25-20d3-48e5-9030-3ec70f401ca1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:24:53.588780 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4443db25-20d3-48e5-9030-3ec70f401ca1 admin admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:24:53.594872 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfbc3b3a-ebed-4515-b0c6-096235369adb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8191d3ccb874529b32f8d0026db629d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '540a32ef88764e8c8d9201ae22f6ddaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:24:53.596062 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbc3b3a-ebed-4515-b0c6-096235369adb service cinder] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:53.596389 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbc3b3a-ebed-4515-b0c6-096235369adb service cinder] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:53.596858 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfbc3b3a-ebed-4515-b0c6-096235369adb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:53.614570 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfbc3b3a-ebed-4515-b0c6-096235369adb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:24:53.629286 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfbc3b3a-ebed-4515-b0c6-096235369adb service cinder] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:24:53.638896 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 957/1892] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 12:24:53 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 12:24:53.926281 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f95310-99b8-4ee2-a229-4c1fff93fdfd None None] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:53.926281 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f95310-99b8-4ee2-a229-4c1fff93fdfd None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:53.926806 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f95310-99b8-4ee2-a229-4c1fff93fdfd None None] PATH_INFO: `/` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:53.928214 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 936/1893] 10.222.0.15 () {54 vars in 921 bytes} [Thu Sep 15 12:24:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:24:53.959127 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58292f88-3948-4af4-86fe-3a3eeaee4a47 None None] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:53.959709 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58292f88-3948-4af4-86fe-3a3eeaee4a47 None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:53.960054 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58292f88-3948-4af4-86fe-3a3eeaee4a47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:54.002169 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.auth.core [None req-58292f88-3948-4af4-86fe-3a3eeaee4a47 None None] MFA Rules not processed for user `eface89e261840a2abb4206be26455c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81449) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:24:54.004053 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-58292f88-3948-4af4-86fe-3a3eeaee4a47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:24:54.117127 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 958/1894] 10.222.0.15 () {58 vars in 1018 bytes} [Thu Sep 15 12:24:53 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:24:54.148628 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f95310-99b8-4ee2-a229-4c1fff93fdfd None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:24:54.148628 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-e5f95310-99b8-4ee2-a229-4c1fff93fdfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:24:54.181084 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f95310-99b8-4ee2-a229-4c1fff93fdfd None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:24:54.188634 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed88aca-c4ab-4540-83e4-96104f21d12a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eface89e261840a2abb4206be26455c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '540a32ef88764e8c8d9201ae22f6ddaf', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:24:54.189480 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed88aca-c4ab-4540-83e4-96104f21d12a service nova] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:54.190444 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed88aca-c4ab-4540-83e4-96104f21d12a service nova] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:54.190928 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed88aca-c4ab-4540-83e4-96104f21d12a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:54.227327 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed88aca-c4ab-4540-83e4-96104f21d12a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:24:54.229302 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed88aca-c4ab-4540-83e4-96104f21d12a service nova] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:24:54.253474 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 937/1895] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 12:24:54 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 12:24:54.282343 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58292f88-3948-4af4-86fe-3a3eeaee4a47 None None] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:24:54.295885 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58292f88-3948-4af4-86fe-3a3eeaee4a47 None None] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:24:54.307602 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf313a1-4e99-49bc-b2a2-719c0979134d tempest-VolumeQuotaClassesTest-348566504 tempest-VolumeQuotaClassesTest-348566504-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fbea441705a4c4092d258fd5bf71cca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef4f5543f5634d1d88240bb77c95fcfc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:24:54.308233 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf313a1-4e99-49bc-b2a2-719c0979134d tempest-VolumeQuotaClassesTest-348566504 tempest-VolumeQuotaClassesTest-348566504-project-admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:54.308520 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf313a1-4e99-49bc-b2a2-719c0979134d tempest-VolumeQuotaClassesTest-348566504 tempest-VolumeQuotaClassesTest-348566504-project-admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:54.308815 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf313a1-4e99-49bc-b2a2-719c0979134d tempest-VolumeQuotaClassesTest-348566504 tempest-VolumeQuotaClassesTest-348566504-project-admin] PATH_INFO: `/v3/domains` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:54.309475 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf313a1-4e99-49bc-b2a2-719c0979134d tempest-VolumeQuotaClassesTest-348566504 tempest-VolumeQuotaClassesTest-348566504-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81449) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:24:54.309779 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf313a1-4e99-49bc-b2a2-719c0979134d tempest-VolumeQuotaClassesTest-348566504 tempest-VolumeQuotaClassesTest-348566504-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:24:54.311553 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf313a1-4e99-49bc-b2a2-719c0979134d tempest-VolumeQuotaClassesTest-348566504 tempest-VolumeQuotaClassesTest-348566504-project-admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:24:54.345121 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 959/1896] 10.222.0.15 () {58 vars in 1153 bytes} [Thu Sep 15 12:24:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:24:54.350288 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed88aca-c4ab-4540-83e4-96104f21d12a service nova] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:24:54.370180 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed88aca-c4ab-4540-83e4-96104f21d12a service nova] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:24:54.394375 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ab1668-a22f-462a-86d1-706fee668d43 tempest-VolumeQuotaClassesTest-348566504 tempest-VolumeQuotaClassesTest-348566504-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fbea441705a4c4092d258fd5bf71cca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef4f5543f5634d1d88240bb77c95fcfc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:24:54.394907 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ab1668-a22f-462a-86d1-706fee668d43 tempest-VolumeQuotaClassesTest-348566504 tempest-VolumeQuotaClassesTest-348566504-project-admin] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:54.395191 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ab1668-a22f-462a-86d1-706fee668d43 tempest-VolumeQuotaClassesTest-348566504 tempest-VolumeQuotaClassesTest-348566504-project-admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:54.395476 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ab1668-a22f-462a-86d1-706fee668d43 tempest-VolumeQuotaClassesTest-348566504 tempest-VolumeQuotaClassesTest-348566504-project-admin] PATH_INFO: `/v3/projects` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:54.396096 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ab1668-a22f-462a-86d1-706fee668d43 tempest-VolumeQuotaClassesTest-348566504 tempest-VolumeQuotaClassesTest-348566504-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:24:54.402227 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ab1668-a22f-462a-86d1-706fee668d43 tempest-VolumeQuotaClassesTest-348566504 tempest-VolumeQuotaClassesTest-348566504-project-admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:24:54.478507 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 938/1897] 10.222.0.15 () {60 vars in 1153 bytes} [Thu Sep 15 12:24:54 2022] POST /identity/v3/projects => generated 353 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 15 12:24:54.480574 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf313a1-4e99-49bc-b2a2-719c0979134d tempest-VolumeQuotaClassesTest-348566504 tempest-VolumeQuotaClassesTest-348566504-project-admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:24:54.512730 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf313a1-4e99-49bc-b2a2-719c0979134d tempest-VolumeQuotaClassesTest-348566504 tempest-VolumeQuotaClassesTest-348566504-project-admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:24:54.609012 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:24:54.670651 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0cbfcb9-2124-4c41-8422-b3391cbba4a6 tempest-VolumeQuotaClassesTest-348566504 tempest-VolumeQuotaClassesTest-348566504-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fbea441705a4c4092d258fd5bf71cca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef4f5543f5634d1d88240bb77c95fcfc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:24:54.671645 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cbfcb9-2124-4c41-8422-b3391cbba4a6 tempest-VolumeQuotaClassesTest-348566504 tempest-VolumeQuotaClassesTest-348566504-project-admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:54.672175 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cbfcb9-2124-4c41-8422-b3391cbba4a6 tempest-VolumeQuotaClassesTest-348566504 tempest-VolumeQuotaClassesTest-348566504-project-admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:54.672520 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cbfcb9-2124-4c41-8422-b3391cbba4a6 tempest-VolumeQuotaClassesTest-348566504 tempest-VolumeQuotaClassesTest-348566504-project-admin] PATH_INFO: `/v3/domains` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:54.673443 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0cbfcb9-2124-4c41-8422-b3391cbba4a6 tempest-VolumeQuotaClassesTest-348566504 tempest-VolumeQuotaClassesTest-348566504-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81449) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:24:54.673932 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0cbfcb9-2124-4c41-8422-b3391cbba4a6 tempest-VolumeQuotaClassesTest-348566504 tempest-VolumeQuotaClassesTest-348566504-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:24:54.675541 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0cbfcb9-2124-4c41-8422-b3391cbba4a6 tempest-VolumeQuotaClassesTest-348566504 tempest-VolumeQuotaClassesTest-348566504-project-admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:24:54.691328 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 960/1898] 10.222.0.15 () {58 vars in 1153 bytes} [Thu Sep 15 12:24:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:24:54.745965 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ab1668-a22f-462a-86d1-706fee668d43 tempest-VolumeQuotaClassesTest-348566504 tempest-VolumeQuotaClassesTest-348566504-project-admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:24:54.758634 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ab1668-a22f-462a-86d1-706fee668d43 tempest-VolumeQuotaClassesTest-348566504 tempest-VolumeQuotaClassesTest-348566504-project-admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:24:54.766872 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70431318-9e43-4e2b-8be9-df22f1c927e7 tempest-VolumeQuotaClassesTest-348566504 tempest-VolumeQuotaClassesTest-348566504-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '2fbea441705a4c4092d258fd5bf71cca', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'ef4f5543f5634d1d88240bb77c95fcfc', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:24:54.767544 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70431318-9e43-4e2b-8be9-df22f1c927e7 tempest-VolumeQuotaClassesTest-348566504 tempest-VolumeQuotaClassesTest-348566504-project-admin] REQUEST_METHOD: `DELETE` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:54.767843 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70431318-9e43-4e2b-8be9-df22f1c927e7 tempest-VolumeQuotaClassesTest-348566504 tempest-VolumeQuotaClassesTest-348566504-project-admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:54.768076 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70431318-9e43-4e2b-8be9-df22f1c927e7 tempest-VolumeQuotaClassesTest-348566504 tempest-VolumeQuotaClassesTest-348566504-project-admin] PATH_INFO: `/v3/projects/bef98c554e294710b06fdcd9421eaf36` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:54.769333 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70431318-9e43-4e2b-8be9-df22f1c927e7 tempest-VolumeQuotaClassesTest-348566504 tempest-VolumeQuotaClassesTest-348566504-project-admin] RBAC: Authorizing `identity:delete_project(project_id=bef98c554e294710b06fdcd9421eaf36)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:24:54.784406 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70431318-9e43-4e2b-8be9-df22f1c927e7 tempest-VolumeQuotaClassesTest-348566504 tempest-VolumeQuotaClassesTest-348566504-project-admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:24:54.908502 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-70431318-9e43-4e2b-8be9-df22f1c927e7 tempest-VolumeQuotaClassesTest-348566504 tempest-VolumeQuotaClassesTest-348566504-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bef98c554e294710b06fdcd9421eaf36', 'request_id': 'req-70431318-9e43-4e2b-8be9-df22f1c927e7'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:24:54.925488 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-70431318-9e43-4e2b-8be9-df22f1c927e7 tempest-VolumeQuotaClassesTest-348566504 tempest-VolumeQuotaClassesTest-348566504-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bef98c554e294710b06fdcd9421eaf36', 'request_id': 'req-70431318-9e43-4e2b-8be9-df22f1c927e7'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:24:54.936009 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-70431318-9e43-4e2b-8be9-df22f1c927e7 tempest-VolumeQuotaClassesTest-348566504 tempest-VolumeQuotaClassesTest-348566504-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bef98c554e294710b06fdcd9421eaf36', 'request_id': 'req-70431318-9e43-4e2b-8be9-df22f1c927e7'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:24:54.951934 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-70431318-9e43-4e2b-8be9-df22f1c927e7 tempest-VolumeQuotaClassesTest-348566504 tempest-VolumeQuotaClassesTest-348566504-project-admin] The token cache is being invalidate because project bef98c554e294710b06fdcd9421eaf36 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81448) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 12:24:54.952310 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-70431318-9e43-4e2b-8be9-df22f1c927e7 tempest-VolumeQuotaClassesTest-348566504 tempest-VolumeQuotaClassesTest-348566504-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:24:54.961486 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-70431318-9e43-4e2b-8be9-df22f1c927e7 tempest-VolumeQuotaClassesTest-348566504 tempest-VolumeQuotaClassesTest-348566504-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:24:54.964547 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 939/1899] 10.222.0.15 () {58 vars in 1233 bytes} [Thu Sep 15 12:24:54 2022] DELETE /identity/v3/projects/bef98c554e294710b06fdcd9421eaf36 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 12:24:55.144677 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0cbfcb9-2124-4c41-8422-b3391cbba4a6 tempest-VolumeQuotaClassesTest-348566504 tempest-VolumeQuotaClassesTest-348566504-project-admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:24:55.146565 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-b0cbfcb9-2124-4c41-8422-b3391cbba4a6 tempest-VolumeQuotaClassesTest-348566504 tempest-VolumeQuotaClassesTest-348566504-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:24:55.257088 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70431318-9e43-4e2b-8be9-df22f1c927e7 tempest-VolumeQuotaClassesTest-348566504 tempest-VolumeQuotaClassesTest-348566504-project-admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:24:55.259292 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-70431318-9e43-4e2b-8be9-df22f1c927e7 tempest-VolumeQuotaClassesTest-348566504 tempest-VolumeQuotaClassesTest-348566504-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:24:55.262374 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0cbfcb9-2124-4c41-8422-b3391cbba4a6 tempest-VolumeQuotaClassesTest-348566504 tempest-VolumeQuotaClassesTest-348566504-project-admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:24:55.265585 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:24:55.339774 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17330d99-ce7c-4fd9-92b4-4119a9773449 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:24:55.340339 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17330d99-ce7c-4fd9-92b4-4119a9773449 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:55.340691 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17330d99-ce7c-4fd9-92b4-4119a9773449 admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:55.341554 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17330d99-ce7c-4fd9-92b4-4119a9773449 admin admin] PATH_INFO: `/v3/users/205b2ef591c641838ca51affce1f4c0c` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:55.343205 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17330d99-ce7c-4fd9-92b4-4119a9773449 admin admin] RBAC: Authorizing `identity:delete_user(user_id=205b2ef591c641838ca51affce1f4c0c)` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:24:55.346193 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17330d99-ce7c-4fd9-92b4-4119a9773449 admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:24:55.365567 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70431318-9e43-4e2b-8be9-df22f1c927e7 tempest-VolumeQuotaClassesTest-348566504 tempest-VolumeQuotaClassesTest-348566504-project-admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:24:55.371208 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a75bb79-cdad-4211-981a-1cc0832a5b2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:24:55.372135 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a75bb79-cdad-4211-981a-1cc0832a5b2a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:55.372615 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a75bb79-cdad-4211-981a-1cc0832a5b2a admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:55.373124 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a75bb79-cdad-4211-981a-1cc0832a5b2a admin admin] PATH_INFO: `/v3/users/19c29ddef4f74d05bd075846d76e0393` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:55.374577 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a75bb79-cdad-4211-981a-1cc0832a5b2a admin admin] RBAC: Authorizing `identity:delete_user(user_id=19c29ddef4f74d05bd075846d76e0393)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:24:55.377055 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a75bb79-cdad-4211-981a-1cc0832a5b2a admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:24:55.421358 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-17330d99-ce7c-4fd9-92b4-4119a9773449 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '205b2ef591c641838ca51affce1f4c0c'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:24:55.436886 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-17330d99-ce7c-4fd9-92b4-4119a9773449 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '205b2ef591c641838ca51affce1f4c0c'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:24:55.463547 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-17330d99-ce7c-4fd9-92b4-4119a9773449 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '205b2ef591c641838ca51affce1f4c0c'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:24:55.464382 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-17330d99-ce7c-4fd9-92b4-4119a9773449 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '205b2ef591c641838ca51affce1f4c0c'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:24:55.476565 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-17330d99-ce7c-4fd9-92b4-4119a9773449 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '205b2ef591c641838ca51affce1f4c0c'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:24:55.481163 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-8a75bb79-cdad-4211-981a-1cc0832a5b2a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '19c29ddef4f74d05bd075846d76e0393'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:24:55.485909 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 961/1900] 10.222.0.15 () {58 vars in 1224 bytes} [Thu Sep 15 12:24:55 2022] DELETE /identity/v3/users/205b2ef591c641838ca51affce1f4c0c => generated 0 bytes in 341 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:24:55.491485 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b80f340-e456-40a7-99aa-f0ebb5131296 None None] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:55.491961 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b80f340-e456-40a7-99aa-f0ebb5131296 None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:55.493405 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b80f340-e456-40a7-99aa-f0ebb5131296 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:55.497250 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-8a75bb79-cdad-4211-981a-1cc0832a5b2a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '19c29ddef4f74d05bd075846d76e0393'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:24:55.513144 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-8a75bb79-cdad-4211-981a-1cc0832a5b2a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '19c29ddef4f74d05bd075846d76e0393'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:24:55.514072 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-8a75bb79-cdad-4211-981a-1cc0832a5b2a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '19c29ddef4f74d05bd075846d76e0393'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:24:55.519008 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:24:55.533099 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-8a75bb79-cdad-4211-981a-1cc0832a5b2a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '19c29ddef4f74d05bd075846d76e0393'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:24:55.539196 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.auth.core [None req-5b80f340-e456-40a7-99aa-f0ebb5131296 None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81449) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:24:55.543487 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-5b80f340-e456-40a7-99aa-f0ebb5131296 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:24:55.546738 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 940/1901] 10.222.0.15 () {58 vars in 1224 bytes} [Thu Sep 15 12:24:55 2022] DELETE /identity/v3/users/19c29ddef4f74d05bd075846d76e0393 => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:24:55.551240 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0555a3-6e3e-4a39-93f6-e95c1af4daa5 None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:55.551533 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0555a3-6e3e-4a39-93f6-e95c1af4daa5 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:55.551812 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0555a3-6e3e-4a39-93f6-e95c1af4daa5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:55.585115 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-3c0555a3-6e3e-4a39-93f6-e95c1af4daa5 None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:24:55.586413 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-3c0555a3-6e3e-4a39-93f6-e95c1af4daa5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:24:55.628135 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 962/1902] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 12:24:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:24:55.643463 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b80f340-e456-40a7-99aa-f0ebb5131296 None None] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:24:55.645442 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-5b80f340-e456-40a7-99aa-f0ebb5131296 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:24:55.666514 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 941/1903] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 12:24:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:24:55.676804 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0555a3-6e3e-4a39-93f6-e95c1af4daa5 None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:24:55.679042 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-3c0555a3-6e3e-4a39-93f6-e95c1af4daa5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:24:55.710220 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:24:55.739964 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b80f340-e456-40a7-99aa-f0ebb5131296 None None] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:24:55.791947 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c0555a3-6e3e-4a39-93f6-e95c1af4daa5 None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:24:55.797025 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1330c0a9-6de2-4fb6-92a8-9ba328dfc585 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfbb4d6809064f9cb9e06626ed7d6814', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '540a32ef88764e8c8d9201ae22f6ddaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:24:55.798514 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1330c0a9-6de2-4fb6-92a8-9ba328dfc585 service neutron] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:55.800395 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1330c0a9-6de2-4fb6-92a8-9ba328dfc585 service neutron] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:55.801031 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1330c0a9-6de2-4fb6-92a8-9ba328dfc585 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:55.812368 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1330c0a9-6de2-4fb6-92a8-9ba328dfc585 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:24:55.814241 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1330c0a9-6de2-4fb6-92a8-9ba328dfc585 service neutron] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:24:55.822055 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756b60dc-125f-4a4b-b4aa-79f0de17daa2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfbb4d6809064f9cb9e06626ed7d6814', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '540a32ef88764e8c8d9201ae22f6ddaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:24:55.823127 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756b60dc-125f-4a4b-b4aa-79f0de17daa2 service neutron] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:55.823127 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756b60dc-125f-4a4b-b4aa-79f0de17daa2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:55.823309 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756b60dc-125f-4a4b-b4aa-79f0de17daa2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:55.829361 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 942/1904] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 12:24:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 15 12:24:55.834740 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-756b60dc-125f-4a4b-b4aa-79f0de17daa2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:24:55.839482 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-756b60dc-125f-4a4b-b4aa-79f0de17daa2 service neutron] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:24:55.851126 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 963/1905] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 12:24:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 208 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 15 12:24:56.582293 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1330c0a9-6de2-4fb6-92a8-9ba328dfc585 service neutron] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:24:56.583842 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-1330c0a9-6de2-4fb6-92a8-9ba328dfc585 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:24:56.590017 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756b60dc-125f-4a4b-b4aa-79f0de17daa2 service neutron] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:24:56.592219 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-756b60dc-125f-4a4b-b4aa-79f0de17daa2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:24:56.602130 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1330c0a9-6de2-4fb6-92a8-9ba328dfc585 service neutron] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:24:56.608223 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c6f525-aee0-48c2-b010-1999d87cac5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:24:56.608988 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c6f525-aee0-48c2-b010-1999d87cac5e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:56.609271 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c6f525-aee0-48c2-b010-1999d87cac5e admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:56.609551 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c6f525-aee0-48c2-b010-1999d87cac5e admin admin] PATH_INFO: `/v3/projects/4696b497d7c54d049e79e5cfe2d000c3` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:56.611009 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c6f525-aee0-48c2-b010-1999d87cac5e admin admin] RBAC: Authorizing `identity:delete_project(project_id=4696b497d7c54d049e79e5cfe2d000c3)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:24:56.613329 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c6f525-aee0-48c2-b010-1999d87cac5e admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:24:56.614625 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756b60dc-125f-4a4b-b4aa-79f0de17daa2 service neutron] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:24:56.634642 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a558d99-6a74-4995-a7db-db32e69484b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:24:56.636815 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a558d99-6a74-4995-a7db-db32e69484b6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:56.637126 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a558d99-6a74-4995-a7db-db32e69484b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:56.637410 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a558d99-6a74-4995-a7db-db32e69484b6 admin admin] PATH_INFO: `/v3/projects/e785b10079b84944b7143167560e8385` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:56.638713 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a558d99-6a74-4995-a7db-db32e69484b6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e785b10079b84944b7143167560e8385)` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:24:56.645866 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a558d99-6a74-4995-a7db-db32e69484b6 admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:24:56.691519 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-60c6f525-aee0-48c2-b010-1999d87cac5e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4696b497d7c54d049e79e5cfe2d000c3', 'request_id': 'req-60c6f525-aee0-48c2-b010-1999d87cac5e'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:24:56.701579 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-60c6f525-aee0-48c2-b010-1999d87cac5e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4696b497d7c54d049e79e5cfe2d000c3', 'request_id': 'req-60c6f525-aee0-48c2-b010-1999d87cac5e'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:24:56.709345 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-60c6f525-aee0-48c2-b010-1999d87cac5e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4696b497d7c54d049e79e5cfe2d000c3', 'request_id': 'req-60c6f525-aee0-48c2-b010-1999d87cac5e'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:24:56.713084 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-8a558d99-6a74-4995-a7db-db32e69484b6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e785b10079b84944b7143167560e8385', 'request_id': 'req-8a558d99-6a74-4995-a7db-db32e69484b6'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:24:56.721414 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-60c6f525-aee0-48c2-b010-1999d87cac5e admin admin] The token cache is being invalidate because project 4696b497d7c54d049e79e5cfe2d000c3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81448) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 12:24:56.721602 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-8a558d99-6a74-4995-a7db-db32e69484b6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e785b10079b84944b7143167560e8385', 'request_id': 'req-8a558d99-6a74-4995-a7db-db32e69484b6'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:24:56.721708 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-60c6f525-aee0-48c2-b010-1999d87cac5e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:24:56.722333 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-60c6f525-aee0-48c2-b010-1999d87cac5e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:24:56.723792 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:24:56.724173 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 943/1906] 10.222.0.15 () {58 vars in 1233 bytes} [Thu Sep 15 12:24:56 2022] DELETE /identity/v3/projects/4696b497d7c54d049e79e5cfe2d000c3 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:24:56.728050 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c6f525-aee0-48c2-b010-1999d87cac5e admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:24:56.729450 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-60c6f525-aee0-48c2-b010-1999d87cac5e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:24:56.730420 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-8a558d99-6a74-4995-a7db-db32e69484b6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e785b10079b84944b7143167560e8385', 'request_id': 'req-8a558d99-6a74-4995-a7db-db32e69484b6'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:24:56.745214 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-8a558d99-6a74-4995-a7db-db32e69484b6 admin admin] The token cache is being invalidate because project e785b10079b84944b7143167560e8385 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81449) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 12:24:56.745654 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-8a558d99-6a74-4995-a7db-db32e69484b6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:24:56.746415 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-8a558d99-6a74-4995-a7db-db32e69484b6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:24:56.750977 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 964/1907] 10.222.0.15 () {58 vars in 1233 bytes} [Thu Sep 15 12:24:56 2022] DELETE /identity/v3/projects/e785b10079b84944b7143167560e8385 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 12:24:56.752078 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a558d99-6a74-4995-a7db-db32e69484b6 admin admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:24:56.754191 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-8a558d99-6a74-4995-a7db-db32e69484b6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:24:56.804336 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c6f525-aee0-48c2-b010-1999d87cac5e admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:24:56.830983 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a558d99-6a74-4995-a7db-db32e69484b6 admin admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:24:56.836665 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55259aab-33b5-4c81-b382-8e5d0691ea53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:24:56.837775 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55259aab-33b5-4c81-b382-8e5d0691ea53 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:56.838132 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55259aab-33b5-4c81-b382-8e5d0691ea53 admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:56.838476 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55259aab-33b5-4c81-b382-8e5d0691ea53 admin admin] PATH_INFO: `/v3/users/bc577218c6ca4d0aa7548023ac2719f9` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:56.839732 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55259aab-33b5-4c81-b382-8e5d0691ea53 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bc577218c6ca4d0aa7548023ac2719f9)` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:24:56.842001 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55259aab-33b5-4c81-b382-8e5d0691ea53 admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:24:56.865251 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:24:56.875659 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a2c73d-f359-4bc1-ad79-022a57878b34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:24:56.876485 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a2c73d-f359-4bc1-ad79-022a57878b34 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:56.876907 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a2c73d-f359-4bc1-ad79-022a57878b34 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:56.877218 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a2c73d-f359-4bc1-ad79-022a57878b34 admin admin] PATH_INFO: `/v3/users/2fbea441705a4c4092d258fd5bf71cca` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:56.878770 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a2c73d-f359-4bc1-ad79-022a57878b34 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2fbea441705a4c4092d258fd5bf71cca)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:24:56.882174 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a2c73d-f359-4bc1-ad79-022a57878b34 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:24:56.894076 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-55259aab-33b5-4c81-b382-8e5d0691ea53 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bc577218c6ca4d0aa7548023ac2719f9'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:24:56.902053 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-55259aab-33b5-4c81-b382-8e5d0691ea53 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bc577218c6ca4d0aa7548023ac2719f9'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:24:56.915126 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-55259aab-33b5-4c81-b382-8e5d0691ea53 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bc577218c6ca4d0aa7548023ac2719f9'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:24:56.916036 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-55259aab-33b5-4c81-b382-8e5d0691ea53 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bc577218c6ca4d0aa7548023ac2719f9'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:24:56.923265 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-55259aab-33b5-4c81-b382-8e5d0691ea53 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bc577218c6ca4d0aa7548023ac2719f9'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:24:56.929653 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 965/1908] 10.222.0.15 () {58 vars in 1224 bytes} [Thu Sep 15 12:24:56 2022] DELETE /identity/v3/users/bc577218c6ca4d0aa7548023ac2719f9 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:24:56.931738 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-f6a2c73d-f359-4bc1-ad79-022a57878b34 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2fbea441705a4c4092d258fd5bf71cca'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:24:56.950405 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-f6a2c73d-f359-4bc1-ad79-022a57878b34 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2fbea441705a4c4092d258fd5bf71cca'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:24:56.957953 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-f6a2c73d-f359-4bc1-ad79-022a57878b34 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2fbea441705a4c4092d258fd5bf71cca'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:24:56.958608 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-f6a2c73d-f359-4bc1-ad79-022a57878b34 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2fbea441705a4c4092d258fd5bf71cca'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:24:56.967764 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-f6a2c73d-f359-4bc1-ad79-022a57878b34 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2fbea441705a4c4092d258fd5bf71cca'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:24:56.975212 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 944/1909] 10.222.0.15 () {58 vars in 1224 bytes} [Thu Sep 15 12:24:56 2022] DELETE /identity/v3/users/2fbea441705a4c4092d258fd5bf71cca => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 12:24:57.323516 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55259aab-33b5-4c81-b382-8e5d0691ea53 admin admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:24:57.325634 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-55259aab-33b5-4c81-b382-8e5d0691ea53 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:24:57.435385 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55259aab-33b5-4c81-b382-8e5d0691ea53 admin admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:24:57.452872 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:24:57.531318 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b98a5a3-a581-4e13-9683-92387eb2cedc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:24:57.532163 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b98a5a3-a581-4e13-9683-92387eb2cedc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:57.532462 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b98a5a3-a581-4e13-9683-92387eb2cedc admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:57.532720 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b98a5a3-a581-4e13-9683-92387eb2cedc admin admin] PATH_INFO: `/v3/projects/ef4f5543f5634d1d88240bb77c95fcfc` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:57.534645 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b98a5a3-a581-4e13-9683-92387eb2cedc admin admin] RBAC: Authorizing `identity:delete_project(project_id=ef4f5543f5634d1d88240bb77c95fcfc)` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:24:57.536706 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b98a5a3-a581-4e13-9683-92387eb2cedc admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:24:57.589438 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a2c73d-f359-4bc1-ad79-022a57878b34 admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:24:57.591628 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-f6a2c73d-f359-4bc1-ad79-022a57878b34 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:24:57.632664 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a2c73d-f359-4bc1-ad79-022a57878b34 admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:24:57.662078 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-3b98a5a3-a581-4e13-9683-92387eb2cedc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ef4f5543f5634d1d88240bb77c95fcfc', 'request_id': 'req-3b98a5a3-a581-4e13-9683-92387eb2cedc'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:24:57.673648 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-3b98a5a3-a581-4e13-9683-92387eb2cedc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ef4f5543f5634d1d88240bb77c95fcfc', 'request_id': 'req-3b98a5a3-a581-4e13-9683-92387eb2cedc'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:24:57.687182 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-3b98a5a3-a581-4e13-9683-92387eb2cedc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ef4f5543f5634d1d88240bb77c95fcfc', 'request_id': 'req-3b98a5a3-a581-4e13-9683-92387eb2cedc'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:24:57.704350 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-3b98a5a3-a581-4e13-9683-92387eb2cedc admin admin] The token cache is being invalidate because project ef4f5543f5634d1d88240bb77c95fcfc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81449) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 12:24:57.704642 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-3b98a5a3-a581-4e13-9683-92387eb2cedc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:24:57.706020 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-3b98a5a3-a581-4e13-9683-92387eb2cedc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:24:57.707794 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 966/1910] 10.222.0.15 () {58 vars in 1233 bytes} [Thu Sep 15 12:24:57 2022] DELETE /identity/v3/projects/ef4f5543f5634d1d88240bb77c95fcfc => generated 0 bytes in 385 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:24:57.712980 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd4f4d0-539e-471f-a447-0fddbf365c2f None None] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:57.713265 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd4f4d0-539e-471f-a447-0fddbf365c2f None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:57.713825 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd4f4d0-539e-471f-a447-0fddbf365c2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:57.754965 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1590b26-4204-4235-9f3c-3126711ac7ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:24:57.755785 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1590b26-4204-4235-9f3c-3126711ac7ea admin admin] REQUEST_METHOD: `DELETE` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:57.756189 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1590b26-4204-4235-9f3c-3126711ac7ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:57.756534 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1590b26-4204-4235-9f3c-3126711ac7ea admin admin] PATH_INFO: `/v3/projects/5bd93e804d234514829f2c01a1541151` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:57.757840 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1590b26-4204-4235-9f3c-3126711ac7ea admin admin] RBAC: Authorizing `identity:delete_project(project_id=5bd93e804d234514829f2c01a1541151)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:24:57.760438 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1590b26-4204-4235-9f3c-3126711ac7ea admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:24:57.761621 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.auth.core [None req-0fd4f4d0-539e-471f-a447-0fddbf365c2f None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81449) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:24:57.765172 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-0fd4f4d0-539e-471f-a447-0fddbf365c2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:24:57.790719 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 967/1911] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 12:24:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:24:57.799385 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d6e3e3-3dbc-4d92-b924-41b934399517 None None] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:57.800091 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d6e3e3-3dbc-4d92-b924-41b934399517 None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:57.800541 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d6e3e3-3dbc-4d92-b924-41b934399517 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:57.830977 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-f1590b26-4204-4235-9f3c-3126711ac7ea admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5bd93e804d234514829f2c01a1541151', 'request_id': 'req-f1590b26-4204-4235-9f3c-3126711ac7ea'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:24:57.833087 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.auth.core [None req-f5d6e3e3-3dbc-4d92-b924-41b934399517 None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81449) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:24:57.835027 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-f5d6e3e3-3dbc-4d92-b924-41b934399517 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:24:57.842857 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-f1590b26-4204-4235-9f3c-3126711ac7ea admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5bd93e804d234514829f2c01a1541151', 'request_id': 'req-f1590b26-4204-4235-9f3c-3126711ac7ea'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:24:57.853459 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-f1590b26-4204-4235-9f3c-3126711ac7ea admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5bd93e804d234514829f2c01a1541151', 'request_id': 'req-f1590b26-4204-4235-9f3c-3126711ac7ea'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:24:57.871831 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-f1590b26-4204-4235-9f3c-3126711ac7ea admin admin] The token cache is being invalidate because project 5bd93e804d234514829f2c01a1541151 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81448) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 12:24:57.872291 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-f1590b26-4204-4235-9f3c-3126711ac7ea admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:24:57.873096 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-f1590b26-4204-4235-9f3c-3126711ac7ea admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:24:57.875319 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 945/1912] 10.222.0.15 () {58 vars in 1233 bytes} [Thu Sep 15 12:24:57 2022] DELETE /identity/v3/projects/5bd93e804d234514829f2c01a1541151 => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:24:57.880347 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9cfb08-5fbd-4415-b4ab-09ae5d131b93 None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:57.880694 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9cfb08-5fbd-4415-b4ab-09ae5d131b93 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:57.881418 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9cfb08-5fbd-4415-b4ab-09ae5d131b93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:57.892274 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:24:57.905555 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-ba9cfb08-5fbd-4415-b4ab-09ae5d131b93 None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:24:57.908164 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-ba9cfb08-5fbd-4415-b4ab-09ae5d131b93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:24:57.953222 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 968/1913] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 12:24:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:24:57.957513 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d6e3e3-3dbc-4d92-b924-41b934399517 None None] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:24:57.981201 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d6e3e3-3dbc-4d92-b924-41b934399517 None None] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:24:57.990439 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83eec62-2211-4287-bc20-d4590dca7592 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:24:57.991163 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83eec62-2211-4287-bc20-d4590dca7592 admin admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:57.991542 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83eec62-2211-4287-bc20-d4590dca7592 admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:57.991923 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83eec62-2211-4287-bc20-d4590dca7592 admin admin] PATH_INFO: `/v3/domains` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:57.992683 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83eec62-2211-4287-bc20-d4590dca7592 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81449) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:24:57.993079 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83eec62-2211-4287-bc20-d4590dca7592 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:24:57.994740 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83eec62-2211-4287-bc20-d4590dca7592 admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:24:58.006723 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 969/1914] 10.222.0.15 () {58 vars in 1153 bytes} [Thu Sep 15 12:24:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:24:58.011369 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83eec62-2211-4287-bc20-d4590dca7592 admin admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:24:58.014013 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 946/1915] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 12:24:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:24:58.021377 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6363bc-6884-4c6a-9c2a-9496e8363ae0 None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:58.021736 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6363bc-6884-4c6a-9c2a-9496e8363ae0 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:58.022081 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6363bc-6884-4c6a-9c2a-9496e8363ae0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:58.022466 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83eec62-2211-4287-bc20-d4590dca7592 admin admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:24:58.028225 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0aef051-b115-414d-b492-84bb62b44929 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:24:58.028856 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0aef051-b115-414d-b492-84bb62b44929 admin admin] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:58.029136 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0aef051-b115-414d-b492-84bb62b44929 admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:58.029406 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0aef051-b115-414d-b492-84bb62b44929 admin admin] PATH_INFO: `/v3/projects` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:58.030197 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0aef051-b115-414d-b492-84bb62b44929 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:24:58.034317 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0aef051-b115-414d-b492-84bb62b44929 admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:24:58.043094 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-8a6363bc-6884-4c6a-9c2a-9496e8363ae0 None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:24:58.044398 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-8a6363bc-6884-4c6a-9c2a-9496e8363ae0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:24:58.053508 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 947/1916] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 12:24:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 12:24:58.057544 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6363bc-6884-4c6a-9c2a-9496e8363ae0 None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:24:58.078294 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6363bc-6884-4c6a-9c2a-9496e8363ae0 None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:24:58.082383 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 970/1917] 10.222.0.15 () {60 vars in 1153 bytes} [Thu Sep 15 12:24:58 2022] POST /identity/v3/projects => generated 379 bytes in 69 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 15 12:24:58.083367 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0aef051-b115-414d-b492-84bb62b44929 admin admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:24:58.089052 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:24:58.151535 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a48da61-4fb8-478f-ade8-3e67ff8c3e8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:24:58.152148 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a48da61-4fb8-478f-ade8-3e67ff8c3e8b admin admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:58.152443 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a48da61-4fb8-478f-ade8-3e67ff8c3e8b admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:58.152740 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a48da61-4fb8-478f-ade8-3e67ff8c3e8b admin admin] PATH_INFO: `/v3/domains` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:58.153410 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a48da61-4fb8-478f-ade8-3e67ff8c3e8b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:24:58.153762 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a48da61-4fb8-478f-ade8-3e67ff8c3e8b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:24:58.155346 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a48da61-4fb8-478f-ade8-3e67ff8c3e8b admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:24:58.158130 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0aef051-b115-414d-b492-84bb62b44929 admin admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:24:58.163007 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1980bd40-f0a2-436f-98c5-2fd23db9e391 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:24:58.163721 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1980bd40-f0a2-436f-98c5-2fd23db9e391 admin admin] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:58.164080 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1980bd40-f0a2-436f-98c5-2fd23db9e391 admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:58.164463 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1980bd40-f0a2-436f-98c5-2fd23db9e391 admin admin] PATH_INFO: `/v3/users` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:58.165323 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1980bd40-f0a2-436f-98c5-2fd23db9e391 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:24:58.167339 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1980bd40-f0a2-436f-98c5-2fd23db9e391 admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:24:58.173747 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 948/1918] 10.222.0.15 () {58 vars in 1153 bytes} [Thu Sep 15 12:24:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 12:24:58.181786 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a48da61-4fb8-478f-ade8-3e67ff8c3e8b admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:24:58.193877 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a48da61-4fb8-478f-ade8-3e67ff8c3e8b admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:24:58.194986 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 971/1919] 10.222.0.15 () {60 vars in 1144 bytes} [Thu Sep 15 12:24:58 2022] POST /identity/v3/users => generated 290 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 12:24:58.199773 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1980bd40-f0a2-436f-98c5-2fd23db9e391 admin admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:24:58.202460 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d9de1c-e364-41e7-90d9-17b49d285653 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:24:58.203376 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d9de1c-e364-41e7-90d9-17b49d285653 admin admin] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:58.203825 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d9de1c-e364-41e7-90d9-17b49d285653 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:58.204479 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d9de1c-e364-41e7-90d9-17b49d285653 admin admin] PATH_INFO: `/v3/projects` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:58.206977 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d9de1c-e364-41e7-90d9-17b49d285653 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:24:58.210318 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d9de1c-e364-41e7-90d9-17b49d285653 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:24:58.211053 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1980bd40-f0a2-436f-98c5-2fd23db9e391 admin admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:24:58.216042 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b9ec15-8579-4d4a-9dd5-3814cec5c3ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:24:58.216660 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b9ec15-8579-4d4a-9dd5-3814cec5c3ed admin admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:58.216979 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b9ec15-8579-4d4a-9dd5-3814cec5c3ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:58.217253 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b9ec15-8579-4d4a-9dd5-3814cec5c3ed admin admin] PATH_INFO: `/v3/roles` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:58.217898 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b9ec15-8579-4d4a-9dd5-3814cec5c3ed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:24:58.219503 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b9ec15-8579-4d4a-9dd5-3814cec5c3ed admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:24:58.232165 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 972/1920] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 12:24:58 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 12:24:58.236034 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b9ec15-8579-4d4a-9dd5-3814cec5c3ed admin admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:24:58.248460 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b9ec15-8579-4d4a-9dd5-3814cec5c3ed admin admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:24:58.250994 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 949/1921] 10.222.0.15 () {60 vars in 1153 bytes} [Thu Sep 15 12:24:58 2022] POST /identity/v3/projects => generated 381 bytes in 72 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 12:24:58.255006 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d9de1c-e364-41e7-90d9-17b49d285653 admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:24:58.255643 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae786fc0-8e65-4286-9560-a18b12b01b24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:24:58.256304 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae786fc0-8e65-4286-9560-a18b12b01b24 admin admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:58.256861 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae786fc0-8e65-4286-9560-a18b12b01b24 admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:58.256861 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae786fc0-8e65-4286-9560-a18b12b01b24 admin admin] PATH_INFO: `/v3/roles` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:58.257516 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae786fc0-8e65-4286-9560-a18b12b01b24 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:24:58.259676 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae786fc0-8e65-4286-9560-a18b12b01b24 admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:24:58.270910 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 973/1922] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 12:24:58 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 15 12:24:58.274939 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae786fc0-8e65-4286-9560-a18b12b01b24 admin admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:24:58.285151 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae786fc0-8e65-4286-9560-a18b12b01b24 admin admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:24:58.321437 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d9de1c-e364-41e7-90d9-17b49d285653 admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:24:58.326030 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73220b3e-346f-4701-9253-84c824509271 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:24:58.326636 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73220b3e-346f-4701-9253-84c824509271 admin admin] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:58.326913 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73220b3e-346f-4701-9253-84c824509271 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:58.327180 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73220b3e-346f-4701-9253-84c824509271 admin admin] PATH_INFO: `/v3/users` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:58.327800 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73220b3e-346f-4701-9253-84c824509271 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:24:58.329718 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73220b3e-346f-4701-9253-84c824509271 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:24:58.343043 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ba42f2-2fd0-49cf-be7a-cdebe4dc2249 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:24:58.343709 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ba42f2-2fd0-49cf-be7a-cdebe4dc2249 admin admin] REQUEST_METHOD: `PUT` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:58.343989 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ba42f2-2fd0-49cf-be7a-cdebe4dc2249 admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:58.344261 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ba42f2-2fd0-49cf-be7a-cdebe4dc2249 admin admin] PATH_INFO: `/v3/projects/6dab91f787bf49198b77b83c5468869f/users/7f894b889a624fb799caec1fd2559203/roles/0679d6628a25438d90dc4bdc0401eda2` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:58.345981 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:24:58.352525 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 950/1923] 10.222.0.15 () {60 vars in 1144 bytes} [Thu Sep 15 12:24:58 2022] POST /identity/v3/users => generated 291 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 15 12:24:58.356012 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73220b3e-346f-4701-9253-84c824509271 admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:24:58.360218 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ba42f2-2fd0-49cf-be7a-cdebe4dc2249 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6dab91f787bf49198b77b83c5468869f, user_id=7f894b889a624fb799caec1fd2559203, role_id=0679d6628a25438d90dc4bdc0401eda2)` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:24:58.362918 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ba42f2-2fd0-49cf-be7a-cdebe4dc2249 admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:24:58.366218 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73220b3e-346f-4701-9253-84c824509271 admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:24:58.372774 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98667060-11db-4487-af0e-3f6aba6a8dee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:24:58.373655 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98667060-11db-4487-af0e-3f6aba6a8dee admin admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:58.373937 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98667060-11db-4487-af0e-3f6aba6a8dee admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:58.374205 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98667060-11db-4487-af0e-3f6aba6a8dee admin admin] PATH_INFO: `/v3/roles` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:58.374871 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98667060-11db-4487-af0e-3f6aba6a8dee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:24:58.376437 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98667060-11db-4487-af0e-3f6aba6a8dee admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:24:58.377244 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 974/1924] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 15 12:24:58 2022] PUT /identity/v3/projects/6dab91f787bf49198b77b83c5468869f/users/7f894b889a624fb799caec1fd2559203/roles/0679d6628a25438d90dc4bdc0401eda2 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 12:24:58.390155 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 951/1925] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 12:24:58 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 15 12:24:58.390319 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288c23fd-f716-4812-9a69-1585ab2610e9 None None] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:58.390319 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288c23fd-f716-4812-9a69-1585ab2610e9 None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:58.390319 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288c23fd-f716-4812-9a69-1585ab2610e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:58.391221 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98667060-11db-4487-af0e-3f6aba6a8dee admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:24:58.401530 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98667060-11db-4487-af0e-3f6aba6a8dee admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:24:58.418616 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.auth.core [None req-288c23fd-f716-4812-9a69-1585ab2610e9 None None] MFA Rules not processed for user `7f894b889a624fb799caec1fd2559203`. Rule list: `[]` (Enabled: `True`). {{(pid=81449) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:24:58.419811 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-288c23fd-f716-4812-9a69-1585ab2610e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:24:58.462790 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7925b24-7dfc-440d-aad4-18182da0f0ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:24:58.463412 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7925b24-7dfc-440d-aad4-18182da0f0ed admin admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:58.463688 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7925b24-7dfc-440d-aad4-18182da0f0ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:58.463955 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7925b24-7dfc-440d-aad4-18182da0f0ed admin admin] PATH_INFO: `/v3/roles` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:58.464740 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7925b24-7dfc-440d-aad4-18182da0f0ed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:24:58.466369 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7925b24-7dfc-440d-aad4-18182da0f0ed admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:24:58.468639 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:24:58.483098 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 952/1926] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 12:24:58 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 15 12:24:58.487302 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7925b24-7dfc-440d-aad4-18182da0f0ed admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:24:58.509705 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7925b24-7dfc-440d-aad4-18182da0f0ed admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:24:58.524594 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 975/1927] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 12:24:58 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 12:24:58.524734 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea9d7d7-47e1-464e-9140-66887d49274b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:24:58.525375 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea9d7d7-47e1-464e-9140-66887d49274b admin admin] REQUEST_METHOD: `PUT` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:58.526125 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea9d7d7-47e1-464e-9140-66887d49274b admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:58.526401 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea9d7d7-47e1-464e-9140-66887d49274b admin admin] PATH_INFO: `/v3/projects/86c4a8545fe24f4f8ca2ef6a019bbf61/users/188eea3c8f7a48cb845a07ce2edcdd45/roles/0679d6628a25438d90dc4bdc0401eda2` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:58.528721 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288c23fd-f716-4812-9a69-1585ab2610e9 None None] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:24:58.541757 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288c23fd-f716-4812-9a69-1585ab2610e9 None None] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:24:58.547175 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78988805-600e-4322-b3c8-3c5b17d864b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:24:58.548019 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78988805-600e-4322-b3c8-3c5b17d864b0 admin admin] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:58.548472 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78988805-600e-4322-b3c8-3c5b17d864b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:58.548942 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78988805-600e-4322-b3c8-3c5b17d864b0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:58.549665 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78988805-600e-4322-b3c8-3c5b17d864b0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:24:58.551656 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78988805-600e-4322-b3c8-3c5b17d864b0 admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:24:58.553943 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea9d7d7-47e1-464e-9140-66887d49274b admin admin] RBAC: Authorizing `identity:create_grant(project_id=86c4a8545fe24f4f8ca2ef6a019bbf61, user_id=188eea3c8f7a48cb845a07ce2edcdd45, role_id=0679d6628a25438d90dc4bdc0401eda2)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:24:58.558952 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea9d7d7-47e1-464e-9140-66887d49274b admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:24:58.578622 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 953/1928] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 15 12:24:58 2022] PUT /identity/v3/projects/86c4a8545fe24f4f8ca2ef6a019bbf61/users/188eea3c8f7a48cb845a07ce2edcdd45/roles/0679d6628a25438d90dc4bdc0401eda2 => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:24:58.590203 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a8fbd4-baae-475c-8205-acdda90b37ee None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:58.590203 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a8fbd4-baae-475c-8205-acdda90b37ee None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:58.590861 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a8fbd4-baae-475c-8205-acdda90b37ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:58.608534 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 976/1929] 10.222.0.15 () {60 vars in 1153 bytes} [Thu Sep 15 12:24:58 2022] POST /identity/v3/projects => generated 377 bytes in 77 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 15 12:24:58.611472 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78988805-600e-4322-b3c8-3c5b17d864b0 admin admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:24:58.664032 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-51a8fbd4-baae-475c-8205-acdda90b37ee None None] MFA Rules not processed for user `188eea3c8f7a48cb845a07ce2edcdd45`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:24:58.668602 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-51a8fbd4-baae-475c-8205-acdda90b37ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:24:58.735798 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78988805-600e-4322-b3c8-3c5b17d864b0 admin admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:24:58.752374 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2e4083-b684-4e84-a1b9-3d0047b28ad1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:24:58.753470 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2e4083-b684-4e84-a1b9-3d0047b28ad1 admin admin] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:58.754287 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2e4083-b684-4e84-a1b9-3d0047b28ad1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:58.754727 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2e4083-b684-4e84-a1b9-3d0047b28ad1 admin admin] PATH_INFO: `/v3/users` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:58.755594 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b2e4083-b684-4e84-a1b9-3d0047b28ad1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:24:58.757829 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b2e4083-b684-4e84-a1b9-3d0047b28ad1 admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:24:58.782902 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 977/1930] 10.222.0.15 () {60 vars in 1144 bytes} [Thu Sep 15 12:24:58 2022] POST /identity/v3/users => generated 289 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 12:24:58.786431 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2e4083-b684-4e84-a1b9-3d0047b28ad1 admin admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:24:58.788660 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 954/1931] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 12:24:58 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 12:24:58.790359 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a8fbd4-baae-475c-8205-acdda90b37ee None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:24:58.800724 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2e4083-b684-4e84-a1b9-3d0047b28ad1 admin admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:24:58.802222 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a8fbd4-baae-475c-8205-acdda90b37ee None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:24:58.806415 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d73ae3-e5ad-4a8f-a353-e17b15fe1db0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:24:58.807219 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d73ae3-e5ad-4a8f-a353-e17b15fe1db0 admin admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:58.807420 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d73ae3-e5ad-4a8f-a353-e17b15fe1db0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:58.807773 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d73ae3-e5ad-4a8f-a353-e17b15fe1db0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:58.808171 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7aeb54-c6f9-400f-816d-50a6dd60db87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:24:58.808561 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d73ae3-e5ad-4a8f-a353-e17b15fe1db0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:24:58.808940 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7aeb54-c6f9-400f-816d-50a6dd60db87 admin admin] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:58.809222 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7aeb54-c6f9-400f-816d-50a6dd60db87 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:58.809599 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7aeb54-c6f9-400f-816d-50a6dd60db87 admin admin] PATH_INFO: `/v3/projects` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:58.810245 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7aeb54-c6f9-400f-816d-50a6dd60db87 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:24:58.810488 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d73ae3-e5ad-4a8f-a353-e17b15fe1db0 admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:24:58.815998 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7aeb54-c6f9-400f-816d-50a6dd60db87 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:24:58.828186 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 978/1932] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 12:24:58 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 15 12:24:58.831860 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d73ae3-e5ad-4a8f-a353-e17b15fe1db0 admin admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:24:58.844355 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d73ae3-e5ad-4a8f-a353-e17b15fe1db0 admin admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:24:58.849594 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76bc9438-0f4b-47f2-b492-313014c8805f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:24:58.850342 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bc9438-0f4b-47f2-b492-313014c8805f admin admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:58.850725 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bc9438-0f4b-47f2-b492-313014c8805f admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:58.851096 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76bc9438-0f4b-47f2-b492-313014c8805f admin admin] PATH_INFO: `/v3/roles` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:58.852397 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76bc9438-0f4b-47f2-b492-313014c8805f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:24:58.854170 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76bc9438-0f4b-47f2-b492-313014c8805f admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:24:58.866298 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 979/1933] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 12:24:58 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 12:24:58.871004 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76bc9438-0f4b-47f2-b492-313014c8805f admin admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:24:58.880052 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 955/1934] 10.222.0.15 () {60 vars in 1153 bytes} [Thu Sep 15 12:24:58 2022] POST /identity/v3/projects => generated 381 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 12:24:58.884425 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76bc9438-0f4b-47f2-b492-313014c8805f admin admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:24:58.884697 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7aeb54-c6f9-400f-816d-50a6dd60db87 admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:24:58.941803 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:24:58.996259 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7aeb54-c6f9-400f-816d-50a6dd60db87 admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:24:59.002546 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67d14dd-07af-4af9-8b03-ab4ba5a1810f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:24:59.003626 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67d14dd-07af-4af9-8b03-ab4ba5a1810f admin admin] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:59.004098 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67d14dd-07af-4af9-8b03-ab4ba5a1810f admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:59.005018 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67d14dd-07af-4af9-8b03-ab4ba5a1810f admin admin] PATH_INFO: `/v3/users` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:59.005572 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67d14dd-07af-4af9-8b03-ab4ba5a1810f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:24:59.007755 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67d14dd-07af-4af9-8b03-ab4ba5a1810f admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:24:59.054034 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db4a084-a9c5-4728-9912-828a7e48f26a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:24:59.055137 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db4a084-a9c5-4728-9912-828a7e48f26a admin admin] REQUEST_METHOD: `PUT` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:59.055461 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db4a084-a9c5-4728-9912-828a7e48f26a admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:59.055687 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db4a084-a9c5-4728-9912-828a7e48f26a admin admin] PATH_INFO: `/v3/projects/64fc22464ba0487c83257064001b9d71/users/c27b01fc280d4a8486fe48b1db159935/roles/0679d6628a25438d90dc4bdc0401eda2` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:59.060816 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 956/1935] 10.222.0.15 () {60 vars in 1144 bytes} [Thu Sep 15 12:24:58 2022] POST /identity/v3/users => generated 297 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 12:24:59.062550 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67d14dd-07af-4af9-8b03-ab4ba5a1810f admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:24:59.072414 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67d14dd-07af-4af9-8b03-ab4ba5a1810f admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:24:59.077783 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e787cb3f-0cb3-446f-a523-931d909a496b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:24:59.080252 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e787cb3f-0cb3-446f-a523-931d909a496b admin admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:59.080603 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e787cb3f-0cb3-446f-a523-931d909a496b admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:59.081066 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e787cb3f-0cb3-446f-a523-931d909a496b admin admin] PATH_INFO: `/v3/roles` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:59.081795 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e787cb3f-0cb3-446f-a523-931d909a496b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:24:59.083735 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e787cb3f-0cb3-446f-a523-931d909a496b admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:24:59.091205 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db4a084-a9c5-4728-9912-828a7e48f26a admin admin] RBAC: Authorizing `identity:create_grant(project_id=64fc22464ba0487c83257064001b9d71, user_id=c27b01fc280d4a8486fe48b1db159935, role_id=0679d6628a25438d90dc4bdc0401eda2)` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:24:59.094536 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db4a084-a9c5-4728-9912-828a7e48f26a admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:24:59.099139 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 957/1936] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 12:24:59 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 12:24:59.101798 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e787cb3f-0cb3-446f-a523-931d909a496b admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:24:59.126831 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 980/1937] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 15 12:24:58 2022] PUT /identity/v3/projects/64fc22464ba0487c83257064001b9d71/users/c27b01fc280d4a8486fe48b1db159935/roles/0679d6628a25438d90dc4bdc0401eda2 => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 12:24:59.130337 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e787cb3f-0cb3-446f-a523-931d909a496b admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:24:59.134617 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5e5317-d764-423c-86d7-ab62d37067dc None None] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:59.134913 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5e5317-d764-423c-86d7-ab62d37067dc None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:59.135188 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5e5317-d764-423c-86d7-ab62d37067dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:59.165257 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:24:59.181050 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.auth.core [None req-4c5e5317-d764-423c-86d7-ab62d37067dc None None] MFA Rules not processed for user `c27b01fc280d4a8486fe48b1db159935`. Rule list: `[]` (Enabled: `True`). {{(pid=81449) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:24:59.194078 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-4c5e5317-d764-423c-86d7-ab62d37067dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:24:59.256839 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f49253-4ea9-46e9-ae1b-ff9e6bcda9b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:24:59.258429 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f49253-4ea9-46e9-ae1b-ff9e6bcda9b2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:59.258920 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f49253-4ea9-46e9-ae1b-ff9e6bcda9b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:59.260153 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f49253-4ea9-46e9-ae1b-ff9e6bcda9b2 admin admin] PATH_INFO: `/v3/projects/3acb32df8ac24988938d5a40f217be2b/users/770286548ebd4d0eb7f696ac9fdb6090/roles/9c769b5c82884766b2c510b1f0f1364c` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:59.281441 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f49253-4ea9-46e9-ae1b-ff9e6bcda9b2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3acb32df8ac24988938d5a40f217be2b, user_id=770286548ebd4d0eb7f696ac9fdb6090, role_id=9c769b5c82884766b2c510b1f0f1364c)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:24:59.286040 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f49253-4ea9-46e9-ae1b-ff9e6bcda9b2 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:24:59.311649 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 958/1938] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 15 12:24:59 2022] PUT /identity/v3/projects/3acb32df8ac24988938d5a40f217be2b/users/770286548ebd4d0eb7f696ac9fdb6090/roles/9c769b5c82884766b2c510b1f0f1364c => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 12:24:59.312938 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f49253-4ea9-46e9-ae1b-ff9e6bcda9b2 admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:24:59.393031 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 981/1939] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 12:24:59 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 265 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 12:24:59.397612 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5e5317-d764-423c-86d7-ab62d37067dc None None] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:24:59.419275 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5e5317-d764-423c-86d7-ab62d37067dc None None] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:24:59.492965 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f49253-4ea9-46e9-ae1b-ff9e6bcda9b2 admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:24:59.536383 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d31f1e-0d4c-459d-8b71-30ada87bf115 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:24:59.537066 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d31f1e-0d4c-459d-8b71-30ada87bf115 admin admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:59.537344 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d31f1e-0d4c-459d-8b71-30ada87bf115 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:59.537700 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d31f1e-0d4c-459d-8b71-30ada87bf115 admin admin] PATH_INFO: `/v3/roles` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:59.538356 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d31f1e-0d4c-459d-8b71-30ada87bf115 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:24:59.540305 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d31f1e-0d4c-459d-8b71-30ada87bf115 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:24:59.558596 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b88befe-fe33-4e36-878e-366e8985e7a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:24:59.561260 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b88befe-fe33-4e36-878e-366e8985e7a9 admin admin] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:59.562416 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b88befe-fe33-4e36-878e-366e8985e7a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:59.562974 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b88befe-fe33-4e36-878e-366e8985e7a9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:59.564236 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b88befe-fe33-4e36-878e-366e8985e7a9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:24:59.568496 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b88befe-fe33-4e36-878e-366e8985e7a9 admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:24:59.580465 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 959/1940] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 12:24:59 2022] GET /identity/v3/roles => generated 1420 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 12:24:59.582508 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d31f1e-0d4c-459d-8b71-30ada87bf115 admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:24:59.595791 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d31f1e-0d4c-459d-8b71-30ada87bf115 admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:24:59.600904 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4820b9ba-aaa7-45aa-bc7a-8851d02f7df3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:24:59.601596 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4820b9ba-aaa7-45aa-bc7a-8851d02f7df3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:59.602350 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4820b9ba-aaa7-45aa-bc7a-8851d02f7df3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:59.602350 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4820b9ba-aaa7-45aa-bc7a-8851d02f7df3 admin admin] PATH_INFO: `/v3/projects/3acb32df8ac24988938d5a40f217be2b/users/770286548ebd4d0eb7f696ac9fdb6090/roles/9c769b5c82884766b2c510b1f0f1364c` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:59.603716 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 982/1941] 10.222.0.15 () {60 vars in 1153 bytes} [Thu Sep 15 12:24:59 2022] POST /identity/v3/projects => generated 377 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 12:24:59.607656 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4820b9ba-aaa7-45aa-bc7a-8851d02f7df3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3acb32df8ac24988938d5a40f217be2b, user_id=770286548ebd4d0eb7f696ac9fdb6090, role_id=9c769b5c82884766b2c510b1f0f1364c)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:24:59.608181 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b88befe-fe33-4e36-878e-366e8985e7a9 admin admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:24:59.610375 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4820b9ba-aaa7-45aa-bc7a-8851d02f7df3 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:24:59.624830 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 960/1942] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 15 12:24:59 2022] PUT /identity/v3/projects/3acb32df8ac24988938d5a40f217be2b/users/770286548ebd4d0eb7f696ac9fdb6090/roles/9c769b5c82884766b2c510b1f0f1364c => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:24:59.643587 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e976fec-7fc3-4e30-9dfd-e1b543b5052c None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:59.643587 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e976fec-7fc3-4e30-9dfd-e1b543b5052c None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:59.643587 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e976fec-7fc3-4e30-9dfd-e1b543b5052c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:59.669710 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:24:59.687055 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-4e976fec-7fc3-4e30-9dfd-e1b543b5052c None None] MFA Rules not processed for user `770286548ebd4d0eb7f696ac9fdb6090`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:24:59.687055 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-4e976fec-7fc3-4e30-9dfd-e1b543b5052c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:24:59.731470 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b88befe-fe33-4e36-878e-366e8985e7a9 admin admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:24:59.772801 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:24:59.774876 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 961/1943] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 12:24:59 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:24:59.788025 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e976fec-7fc3-4e30-9dfd-e1b543b5052c None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:24:59.789883 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-4e976fec-7fc3-4e30-9dfd-e1b543b5052c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:24:59.809754 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c180f1d7-4aa1-4cb1-b8bf-30373d6452c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:24:59.810523 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c180f1d7-4aa1-4cb1-b8bf-30373d6452c0 admin admin] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:59.810804 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c180f1d7-4aa1-4cb1-b8bf-30373d6452c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:59.811079 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c180f1d7-4aa1-4cb1-b8bf-30373d6452c0 admin admin] PATH_INFO: `/v3/users` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:59.811735 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c180f1d7-4aa1-4cb1-b8bf-30373d6452c0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:24:59.813708 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c180f1d7-4aa1-4cb1-b8bf-30373d6452c0 admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:24:59.842693 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 983/1944] 10.222.0.15 () {60 vars in 1144 bytes} [Thu Sep 15 12:24:59 2022] POST /identity/v3/users => generated 295 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 12:24:59.846716 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c180f1d7-4aa1-4cb1-b8bf-30373d6452c0 admin admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:24:59.863599 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c180f1d7-4aa1-4cb1-b8bf-30373d6452c0 admin admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:24:59.868559 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e976fec-7fc3-4e30-9dfd-e1b543b5052c None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:24:59.870636 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9108e862-0f1f-4bec-88b7-688a9a5af9e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:24:59.871406 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9108e862-0f1f-4bec-88b7-688a9a5af9e1 admin admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:59.871691 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9108e862-0f1f-4bec-88b7-688a9a5af9e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:59.872261 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9108e862-0f1f-4bec-88b7-688a9a5af9e1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:59.872489 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de909d0d-44a5-4b72-a390-7e55e1e38069 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8191d3ccb874529b32f8d0026db629d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '540a32ef88764e8c8d9201ae22f6ddaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:24:59.873279 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de909d0d-44a5-4b72-a390-7e55e1e38069 service cinder] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:59.873464 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9108e862-0f1f-4bec-88b7-688a9a5af9e1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:24:59.873560 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de909d0d-44a5-4b72-a390-7e55e1e38069 service cinder] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:59.874310 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de909d0d-44a5-4b72-a390-7e55e1e38069 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:59.875355 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9108e862-0f1f-4bec-88b7-688a9a5af9e1 admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:24:59.897044 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de909d0d-44a5-4b72-a390-7e55e1e38069 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:24:59.897044 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de909d0d-44a5-4b72-a390-7e55e1e38069 service cinder] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:24:59.897242 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 984/1945] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 12:24:59 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 15 12:24:59.910725 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9108e862-0f1f-4bec-88b7-688a9a5af9e1 admin admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:24:59.915388 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 962/1946] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 12:24:59 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 15 12:24:59.928389 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9108e862-0f1f-4bec-88b7-688a9a5af9e1 admin admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:24:59.934727 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd0b5ea-3685-482b-ab01-c810320ff406 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:24:59.935676 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd0b5ea-3685-482b-ab01-c810320ff406 admin admin] REQUEST_METHOD: `PUT` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:24:59.936128 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd0b5ea-3685-482b-ab01-c810320ff406 admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:24:59.936265 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd0b5ea-3685-482b-ab01-c810320ff406 admin admin] PATH_INFO: `/v3/projects/7156351a003c4b9fa185fc09144746cd/users/702a92cca62b4219a6cfc86d766be78e/roles/9c769b5c82884766b2c510b1f0f1364c` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:24:59.952469 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd0b5ea-3685-482b-ab01-c810320ff406 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7156351a003c4b9fa185fc09144746cd, user_id=702a92cca62b4219a6cfc86d766be78e, role_id=9c769b5c82884766b2c510b1f0f1364c)` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:24:59.960493 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd0b5ea-3685-482b-ab01-c810320ff406 admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:24:59.996170 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 985/1947] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 15 12:24:59 2022] PUT /identity/v3/projects/7156351a003c4b9fa185fc09144746cd/users/702a92cca62b4219a6cfc86d766be78e/roles/9c769b5c82884766b2c510b1f0f1364c => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:24:59.997189 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de909d0d-44a5-4b72-a390-7e55e1e38069 service cinder] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:25:00.014534 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de909d0d-44a5-4b72-a390-7e55e1e38069 service cinder] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:25:00.119994 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0435d1fa-1984-4af1-84d6-ab85e8a1610d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:25:00.120384 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0435d1fa-1984-4af1-84d6-ab85e8a1610d admin admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:25:00.120749 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0435d1fa-1984-4af1-84d6-ab85e8a1610d admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:25:00.121120 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0435d1fa-1984-4af1-84d6-ab85e8a1610d admin admin] PATH_INFO: `/v3/roles` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:25:00.122140 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0435d1fa-1984-4af1-84d6-ab85e8a1610d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:25:00.124017 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0435d1fa-1984-4af1-84d6-ab85e8a1610d admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:25:00.143608 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 963/1948] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 12:24:59 2022] GET /identity/v3/roles => generated 1420 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 15 12:25:00.144851 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd0b5ea-3685-482b-ab01-c810320ff406 admin admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:25:00.164588 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd0b5ea-3685-482b-ab01-c810320ff406 admin admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:25:00.174551 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594af7c4-bce7-49ce-b66c-269332f99678 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:25:00.175242 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594af7c4-bce7-49ce-b66c-269332f99678 admin admin] REQUEST_METHOD: `PUT` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:25:00.175553 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594af7c4-bce7-49ce-b66c-269332f99678 admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:25:00.175966 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594af7c4-bce7-49ce-b66c-269332f99678 admin admin] PATH_INFO: `/v3/projects/7156351a003c4b9fa185fc09144746cd/users/702a92cca62b4219a6cfc86d766be78e/roles/9c769b5c82884766b2c510b1f0f1364c` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:25:00.178548 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594af7c4-bce7-49ce-b66c-269332f99678 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7156351a003c4b9fa185fc09144746cd, user_id=702a92cca62b4219a6cfc86d766be78e, role_id=9c769b5c82884766b2c510b1f0f1364c)` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:25:00.181662 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594af7c4-bce7-49ce-b66c-269332f99678 admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:25:00.206620 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 986/1949] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 15 12:25:00 2022] PUT /identity/v3/projects/7156351a003c4b9fa185fc09144746cd/users/702a92cca62b4219a6cfc86d766be78e/roles/9c769b5c82884766b2c510b1f0f1364c => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:25:00.215457 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a568a1aa-c154-4db1-85a2-2d7106013131 None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:25:00.215851 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a568a1aa-c154-4db1-85a2-2d7106013131 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:25:00.216165 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a568a1aa-c154-4db1-85a2-2d7106013131 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:25:00.255359 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-a568a1aa-c154-4db1-85a2-2d7106013131 None None] MFA Rules not processed for user `702a92cca62b4219a6cfc86d766be78e`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:25:00.257044 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-a568a1aa-c154-4db1-85a2-2d7106013131 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:25:00.299922 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:25:00.348229 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 964/1950] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 12:25:00 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:25:00.361749 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594af7c4-bce7-49ce-b66c-269332f99678 admin admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:25:00.462063 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594af7c4-bce7-49ce-b66c-269332f99678 admin admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:25:00.471332 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43c5fa2-80c0-413c-a8bd-50e14363fc68 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8191d3ccb874529b32f8d0026db629d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '540a32ef88764e8c8d9201ae22f6ddaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:25:00.471332 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43c5fa2-80c0-413c-a8bd-50e14363fc68 service cinder] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:25:00.471332 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43c5fa2-80c0-413c-a8bd-50e14363fc68 service cinder] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:25:00.471332 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43c5fa2-80c0-413c-a8bd-50e14363fc68 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:25:00.494296 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c43c5fa2-80c0-413c-a8bd-50e14363fc68 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:25:00.495796 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c43c5fa2-80c0-413c-a8bd-50e14363fc68 service cinder] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:25:00.513225 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a568a1aa-c154-4db1-85a2-2d7106013131 None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:25:00.519287 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 987/1951] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 12:25:00 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 12:25:00.615030 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43c5fa2-80c0-413c-a8bd-50e14363fc68 service cinder] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:25:00.626985 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43c5fa2-80c0-413c-a8bd-50e14363fc68 service cinder] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:25:00.631746 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5372c6-9626-4496-8459-08b24857cc78 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8191d3ccb874529b32f8d0026db629d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '540a32ef88764e8c8d9201ae22f6ddaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:25:00.632402 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5372c6-9626-4496-8459-08b24857cc78 service cinder] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:25:00.632684 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5372c6-9626-4496-8459-08b24857cc78 service cinder] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:25:00.633562 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5372c6-9626-4496-8459-08b24857cc78 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:25:00.667155 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a568a1aa-c154-4db1-85a2-2d7106013131 None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:25:00.673410 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde53657-e512-4b8e-a4b4-ec0b01bcdfe5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:25:00.674721 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde53657-e512-4b8e-a4b4-ec0b01bcdfe5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:25:00.675529 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde53657-e512-4b8e-a4b4-ec0b01bcdfe5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:25:00.676015 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde53657-e512-4b8e-a4b4-ec0b01bcdfe5 admin admin] PATH_INFO: `/v3/users/188eea3c8f7a48cb845a07ce2edcdd45` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:25:00.677562 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde53657-e512-4b8e-a4b4-ec0b01bcdfe5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=188eea3c8f7a48cb845a07ce2edcdd45)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:25:00.681642 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde53657-e512-4b8e-a4b4-ec0b01bcdfe5 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:25:00.750883 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5372c6-9626-4496-8459-08b24857cc78 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:25:00.753158 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5372c6-9626-4496-8459-08b24857cc78 service cinder] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:25:00.787806 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 988/1952] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 12:25:00 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 15 12:25:00.797624 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-cde53657-e512-4b8e-a4b4-ec0b01bcdfe5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '188eea3c8f7a48cb845a07ce2edcdd45'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:25:00.817404 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-cde53657-e512-4b8e-a4b4-ec0b01bcdfe5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '188eea3c8f7a48cb845a07ce2edcdd45'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:25:00.831200 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-cde53657-e512-4b8e-a4b4-ec0b01bcdfe5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '188eea3c8f7a48cb845a07ce2edcdd45'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:25:00.833044 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-cde53657-e512-4b8e-a4b4-ec0b01bcdfe5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '188eea3c8f7a48cb845a07ce2edcdd45'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:25:00.851133 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-cde53657-e512-4b8e-a4b4-ec0b01bcdfe5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '188eea3c8f7a48cb845a07ce2edcdd45'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:25:00.869740 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 965/1953] 10.222.0.15 () {58 vars in 1224 bytes} [Thu Sep 15 12:25:00 2022] DELETE /identity/v3/users/188eea3c8f7a48cb845a07ce2edcdd45 => generated 0 bytes in 358 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:25:00.874709 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece2048f-8a92-4cf4-b7e7-87c8bdd23f89 None None] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:25:00.875006 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece2048f-8a92-4cf4-b7e7-87c8bdd23f89 None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:25:00.878731 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece2048f-8a92-4cf4-b7e7-87c8bdd23f89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:25:00.925460 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.auth.core [None req-ece2048f-8a92-4cf4-b7e7-87c8bdd23f89 None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81449) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:25:00.927244 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-ece2048f-8a92-4cf4-b7e7-87c8bdd23f89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:25:00.971441 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:25:01.054453 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 989/1954] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 12:25:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:25:01.073491 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde53657-e512-4b8e-a4b4-ec0b01bcdfe5 admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:25:01.078171 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-cde53657-e512-4b8e-a4b4-ec0b01bcdfe5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:25:01.155042 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:25:01.186867 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde53657-e512-4b8e-a4b4-ec0b01bcdfe5 admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:25:01.196428 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef94fff-2b65-44e2-a2fa-31555ee3f23a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfbb4d6809064f9cb9e06626ed7d6814', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '540a32ef88764e8c8d9201ae22f6ddaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:25:01.197536 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef94fff-2b65-44e2-a2fa-31555ee3f23a service neutron] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:25:01.198057 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef94fff-2b65-44e2-a2fa-31555ee3f23a service neutron] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:25:01.198511 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef94fff-2b65-44e2-a2fa-31555ee3f23a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:25:01.218368 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef94fff-2b65-44e2-a2fa-31555ee3f23a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:25:01.220717 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef94fff-2b65-44e2-a2fa-31555ee3f23a service neutron] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:25:01.236025 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 966/1955] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 12:25:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 12:25:02.160068 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece2048f-8a92-4cf4-b7e7-87c8bdd23f89 None None] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:25:02.162340 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-ece2048f-8a92-4cf4-b7e7-87c8bdd23f89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:25:02.176228 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece2048f-8a92-4cf4-b7e7-87c8bdd23f89 None None] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:25:02.191106 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a944feb3-8e0c-43a8-a011-4cb8f4f77b7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:25:02.191791 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a944feb3-8e0c-43a8-a011-4cb8f4f77b7b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:25:02.192067 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a944feb3-8e0c-43a8-a011-4cb8f4f77b7b admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:25:02.192343 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a944feb3-8e0c-43a8-a011-4cb8f4f77b7b admin admin] PATH_INFO: `/v3/projects/86c4a8545fe24f4f8ca2ef6a019bbf61` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:25:02.193744 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a944feb3-8e0c-43a8-a011-4cb8f4f77b7b admin admin] RBAC: Authorizing `identity:delete_project(project_id=86c4a8545fe24f4f8ca2ef6a019bbf61)` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:25:02.196434 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a944feb3-8e0c-43a8-a011-4cb8f4f77b7b admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:25:02.247923 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-a944feb3-8e0c-43a8-a011-4cb8f4f77b7b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '86c4a8545fe24f4f8ca2ef6a019bbf61', 'request_id': 'req-a944feb3-8e0c-43a8-a011-4cb8f4f77b7b'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:25:02.257372 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-a944feb3-8e0c-43a8-a011-4cb8f4f77b7b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '86c4a8545fe24f4f8ca2ef6a019bbf61', 'request_id': 'req-a944feb3-8e0c-43a8-a011-4cb8f4f77b7b'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:25:02.265094 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-a944feb3-8e0c-43a8-a011-4cb8f4f77b7b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '86c4a8545fe24f4f8ca2ef6a019bbf61', 'request_id': 'req-a944feb3-8e0c-43a8-a011-4cb8f4f77b7b'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:25:02.277217 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-a944feb3-8e0c-43a8-a011-4cb8f4f77b7b admin admin] The token cache is being invalidate because project 86c4a8545fe24f4f8ca2ef6a019bbf61 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81449) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 12:25:02.277629 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-a944feb3-8e0c-43a8-a011-4cb8f4f77b7b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:25:02.278363 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-a944feb3-8e0c-43a8-a011-4cb8f4f77b7b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:25:02.280152 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 990/1956] 10.222.0.15 () {58 vars in 1233 bytes} [Thu Sep 15 12:25:02 2022] DELETE /identity/v3/projects/86c4a8545fe24f4f8ca2ef6a019bbf61 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:25:02.284048 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef94fff-2b65-44e2-a2fa-31555ee3f23a service neutron] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:25:02.294192 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef94fff-2b65-44e2-a2fa-31555ee3f23a service neutron] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:25:02.394465 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9b9943-6a38-46a7-be45-56861e0afb35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:25:02.395121 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9b9943-6a38-46a7-be45-56861e0afb35 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:25:02.395403 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9b9943-6a38-46a7-be45-56861e0afb35 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:25:02.395676 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9b9943-6a38-46a7-be45-56861e0afb35 admin admin] PATH_INFO: `/v3/users/770286548ebd4d0eb7f696ac9fdb6090` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:25:02.396911 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9b9943-6a38-46a7-be45-56861e0afb35 admin admin] RBAC: Authorizing `identity:delete_user(user_id=770286548ebd4d0eb7f696ac9fdb6090)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:25:02.398994 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9b9943-6a38-46a7-be45-56861e0afb35 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:25:02.469920 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-2a9b9943-6a38-46a7-be45-56861e0afb35 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '770286548ebd4d0eb7f696ac9fdb6090'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:25:02.487537 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-2a9b9943-6a38-46a7-be45-56861e0afb35 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '770286548ebd4d0eb7f696ac9fdb6090'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:25:02.500489 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-2a9b9943-6a38-46a7-be45-56861e0afb35 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '770286548ebd4d0eb7f696ac9fdb6090'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:25:02.501274 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-2a9b9943-6a38-46a7-be45-56861e0afb35 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '770286548ebd4d0eb7f696ac9fdb6090'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:25:02.509875 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-2a9b9943-6a38-46a7-be45-56861e0afb35 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '770286548ebd4d0eb7f696ac9fdb6090'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:25:02.516824 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 967/1957] 10.222.0.15 () {58 vars in 1224 bytes} [Thu Sep 15 12:25:02 2022] DELETE /identity/v3/users/770286548ebd4d0eb7f696ac9fdb6090 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:25:02.860235 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a944feb3-8e0c-43a8-a011-4cb8f4f77b7b admin admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:25:02.861838 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-a944feb3-8e0c-43a8-a011-4cb8f4f77b7b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:25:02.877373 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a944feb3-8e0c-43a8-a011-4cb8f4f77b7b admin admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:25:02.949357 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ac26dd-7319-4bfe-bae8-20446ed38f5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:25:02.950420 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ac26dd-7319-4bfe-bae8-20446ed38f5e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:25:02.950846 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ac26dd-7319-4bfe-bae8-20446ed38f5e admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:25:02.951161 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ac26dd-7319-4bfe-bae8-20446ed38f5e admin admin] PATH_INFO: `/v3/projects/3acb32df8ac24988938d5a40f217be2b` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:25:02.952627 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ac26dd-7319-4bfe-bae8-20446ed38f5e admin admin] RBAC: Authorizing `identity:delete_project(project_id=3acb32df8ac24988938d5a40f217be2b)` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:25:02.962311 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ac26dd-7319-4bfe-bae8-20446ed38f5e admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:25:02.998463 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06857fa-7caa-482d-854c-2fcb1f19866c None None] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:25:02.998747 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06857fa-7caa-482d-854c-2fcb1f19866c None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:25:02.999021 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06857fa-7caa-482d-854c-2fcb1f19866c None None] PATH_INFO: `/` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:25:03.000409 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 968/1958] 10.222.0.15 () {54 vars in 921 bytes} [Thu Sep 15 12:25:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Sep 15 12:25:03.006817 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32692870-d7ee-4468-a2e1-2e1751f10eaa None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:25:03.007151 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32692870-d7ee-4468-a2e1-2e1751f10eaa None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:25:03.007426 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32692870-d7ee-4468-a2e1-2e1751f10eaa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:25:03.016345 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:25:03.027628 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-83ac26dd-7319-4bfe-bae8-20446ed38f5e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3acb32df8ac24988938d5a40f217be2b', 'request_id': 'req-83ac26dd-7319-4bfe-bae8-20446ed38f5e'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:25:03.029646 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-32692870-d7ee-4468-a2e1-2e1751f10eaa None None] MFA Rules not processed for user `eface89e261840a2abb4206be26455c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:25:03.030873 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-32692870-d7ee-4468-a2e1-2e1751f10eaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:25:03.036039 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-83ac26dd-7319-4bfe-bae8-20446ed38f5e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3acb32df8ac24988938d5a40f217be2b', 'request_id': 'req-83ac26dd-7319-4bfe-bae8-20446ed38f5e'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:25:03.043013 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-83ac26dd-7319-4bfe-bae8-20446ed38f5e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3acb32df8ac24988938d5a40f217be2b', 'request_id': 'req-83ac26dd-7319-4bfe-bae8-20446ed38f5e'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:25:03.055783 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-83ac26dd-7319-4bfe-bae8-20446ed38f5e admin admin] The token cache is being invalidate because project 3acb32df8ac24988938d5a40f217be2b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81449) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 12:25:03.056460 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-83ac26dd-7319-4bfe-bae8-20446ed38f5e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:25:03.057151 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-83ac26dd-7319-4bfe-bae8-20446ed38f5e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:25:03.059219 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 991/1959] 10.222.0.15 () {58 vars in 1233 bytes} [Thu Sep 15 12:25:02 2022] DELETE /identity/v3/projects/3acb32df8ac24988938d5a40f217be2b => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:25:03.064169 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c7fd57-01ae-4a9a-b22d-c6273aca683e None None] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:25:03.064867 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c7fd57-01ae-4a9a-b22d-c6273aca683e None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:25:03.064867 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c7fd57-01ae-4a9a-b22d-c6273aca683e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:25:03.089231 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.auth.core [None req-f4c7fd57-01ae-4a9a-b22d-c6273aca683e None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81449) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:25:03.090498 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-f4c7fd57-01ae-4a9a-b22d-c6273aca683e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:25:03.091079 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:25:03.109535 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 969/1960] 10.222.0.15 () {58 vars in 1018 bytes} [Thu Sep 15 12:25:03 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 12:25:03.164658 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 992/1961] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 12:25:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:25:03.172211 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035aecdd-6c10-40e2-b5fe-d832835809d0 None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:25:03.172723 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035aecdd-6c10-40e2-b5fe-d832835809d0 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:25:03.173171 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035aecdd-6c10-40e2-b5fe-d832835809d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:25:03.214699 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-035aecdd-6c10-40e2-b5fe-d832835809d0 None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:25:03.216102 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-035aecdd-6c10-40e2-b5fe-d832835809d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:25:03.228197 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 970/1962] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 12:25:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:25:03.232484 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c7fd57-01ae-4a9a-b22d-c6273aca683e None None] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:25:03.249777 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c7fd57-01ae-4a9a-b22d-c6273aca683e None None] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:25:03.255896 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d37a3b-3ccd-4b2c-9dcd-a8b8c6d49d44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:25:03.256527 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d37a3b-3ccd-4b2c-9dcd-a8b8c6d49d44 admin admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:25:03.256901 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d37a3b-3ccd-4b2c-9dcd-a8b8c6d49d44 admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:25:03.257175 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d37a3b-3ccd-4b2c-9dcd-a8b8c6d49d44 admin admin] PATH_INFO: `/v3/domains` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:25:03.258372 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d37a3b-3ccd-4b2c-9dcd-a8b8c6d49d44 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81449) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:25:03.258676 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d37a3b-3ccd-4b2c-9dcd-a8b8c6d49d44 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:25:03.260788 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d37a3b-3ccd-4b2c-9dcd-a8b8c6d49d44 admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:25:03.303229 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 993/1963] 10.222.0.15 () {58 vars in 1153 bytes} [Thu Sep 15 12:25:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 12:25:03.309272 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035aecdd-6c10-40e2-b5fe-d832835809d0 None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:25:03.320831 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035aecdd-6c10-40e2-b5fe-d832835809d0 None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:25:03.333461 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d37a3b-3ccd-4b2c-9dcd-a8b8c6d49d44 admin admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:25:03.342137 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-76d37a3b-3ccd-4b2c-9dcd-a8b8c6d49d44 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:25:03.349511 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11698341-9601-4dbe-8ad1-11f80b8a2769 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:25:03.351286 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11698341-9601-4dbe-8ad1-11f80b8a2769 admin admin] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:25:03.351608 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11698341-9601-4dbe-8ad1-11f80b8a2769 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:25:03.351883 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11698341-9601-4dbe-8ad1-11f80b8a2769 admin admin] PATH_INFO: `/v3/projects` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:25:03.353710 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11698341-9601-4dbe-8ad1-11f80b8a2769 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:25:03.355652 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11698341-9601-4dbe-8ad1-11f80b8a2769 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:25:03.391572 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d37a3b-3ccd-4b2c-9dcd-a8b8c6d49d44 admin admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:25:03.403100 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e7fa92-2fb9-431c-9c8f-fc3a9a8ac2ed service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eface89e261840a2abb4206be26455c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '540a32ef88764e8c8d9201ae22f6ddaf', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:25:03.403793 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e7fa92-2fb9-431c-9c8f-fc3a9a8ac2ed service nova] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:25:03.415866 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e7fa92-2fb9-431c-9c8f-fc3a9a8ac2ed service nova] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:25:03.416312 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e7fa92-2fb9-431c-9c8f-fc3a9a8ac2ed service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:25:03.441990 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e7fa92-2fb9-431c-9c8f-fc3a9a8ac2ed service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:25:03.443457 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 971/1964] 10.222.0.15 () {60 vars in 1153 bytes} [Thu Sep 15 12:25:03 2022] POST /identity/v3/projects => generated 385 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 12:25:03.447947 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e7fa92-2fb9-431c-9c8f-fc3a9a8ac2ed service nova] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:25:03.460781 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11698341-9601-4dbe-8ad1-11f80b8a2769 admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:25:03.461405 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 994/1965] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 12:25:03 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 12:25:03.572314 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11698341-9601-4dbe-8ad1-11f80b8a2769 admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:25:03.581824 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1b3c42-f494-4a2b-a59b-b8f900b154e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:25:03.583376 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1b3c42-f494-4a2b-a59b-b8f900b154e0 admin admin] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:25:03.583895 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1b3c42-f494-4a2b-a59b-b8f900b154e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:25:03.584390 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1b3c42-f494-4a2b-a59b-b8f900b154e0 admin admin] PATH_INFO: `/v3/users` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:25:03.585333 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc1b3c42-f494-4a2b-a59b-b8f900b154e0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:25:03.589789 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc1b3c42-f494-4a2b-a59b-b8f900b154e0 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:25:03.646823 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 972/1966] 10.222.0.15 () {60 vars in 1144 bytes} [Thu Sep 15 12:25:03 2022] POST /identity/v3/users => generated 293 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 12:25:03.651283 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e7fa92-2fb9-431c-9c8f-fc3a9a8ac2ed service nova] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:25:03.664955 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e7fa92-2fb9-431c-9c8f-fc3a9a8ac2ed service nova] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:25:03.673719 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645cfdd7-18b7-449e-87bd-1bcb683e6252 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:25:03.673719 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645cfdd7-18b7-449e-87bd-1bcb683e6252 admin admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:25:03.673719 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645cfdd7-18b7-449e-87bd-1bcb683e6252 admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:25:03.673719 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645cfdd7-18b7-449e-87bd-1bcb683e6252 admin admin] PATH_INFO: `/v3/roles` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:25:03.674071 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-645cfdd7-18b7-449e-87bd-1bcb683e6252 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:25:03.674071 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-645cfdd7-18b7-449e-87bd-1bcb683e6252 admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:25:03.686935 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 995/1967] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 12:25:03 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 12:25:03.691101 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1b3c42-f494-4a2b-a59b-b8f900b154e0 admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:25:03.706888 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1b3c42-f494-4a2b-a59b-b8f900b154e0 admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:25:03.713988 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503eda20-6850-43ce-8e51-ceafd16bd6d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:25:03.714701 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503eda20-6850-43ce-8e51-ceafd16bd6d0 admin admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:25:03.715195 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503eda20-6850-43ce-8e51-ceafd16bd6d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:25:03.715543 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503eda20-6850-43ce-8e51-ceafd16bd6d0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:25:03.716276 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503eda20-6850-43ce-8e51-ceafd16bd6d0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:25:03.718221 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503eda20-6850-43ce-8e51-ceafd16bd6d0 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:25:03.732540 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 973/1968] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 12:25:03 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 12:25:03.737610 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645cfdd7-18b7-449e-87bd-1bcb683e6252 admin admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:25:03.755340 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645cfdd7-18b7-449e-87bd-1bcb683e6252 admin admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:25:03.761124 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e937817-ea08-4d94-9e2e-6af1ab9b6886 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:25:03.761813 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e937817-ea08-4d94-9e2e-6af1ab9b6886 admin admin] REQUEST_METHOD: `PUT` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:25:03.762093 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e937817-ea08-4d94-9e2e-6af1ab9b6886 admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:25:03.762366 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e937817-ea08-4d94-9e2e-6af1ab9b6886 admin admin] PATH_INFO: `/v3/projects/b230bd05a52240d8a4c8557d2d3bc5e1/users/d63ca125537749399da8a22568641e13/roles/0679d6628a25438d90dc4bdc0401eda2` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:25:03.782807 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e937817-ea08-4d94-9e2e-6af1ab9b6886 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b230bd05a52240d8a4c8557d2d3bc5e1, user_id=d63ca125537749399da8a22568641e13, role_id=0679d6628a25438d90dc4bdc0401eda2)` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:25:03.785673 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e937817-ea08-4d94-9e2e-6af1ab9b6886 admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:25:03.801479 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 996/1969] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 15 12:25:03 2022] PUT /identity/v3/projects/b230bd05a52240d8a4c8557d2d3bc5e1/users/d63ca125537749399da8a22568641e13/roles/0679d6628a25438d90dc4bdc0401eda2 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:25:03.811637 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f85a6d-0f45-4520-848a-45e81821d856 None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:25:03.811923 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f85a6d-0f45-4520-848a-45e81821d856 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:25:03.812313 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f85a6d-0f45-4520-848a-45e81821d856 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:25:03.849696 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-38f85a6d-0f45-4520-848a-45e81821d856 None None] MFA Rules not processed for user `d63ca125537749399da8a22568641e13`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:25:03.851689 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-38f85a6d-0f45-4520-848a-45e81821d856 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:25:03.936285 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 974/1970] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 12:25:03 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:25:03.940392 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e937817-ea08-4d94-9e2e-6af1ab9b6886 admin admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:25:04.031476 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e937817-ea08-4d94-9e2e-6af1ab9b6886 admin admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:25:04.036089 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d691e5-713c-4030-bbc6-2ff82a64fed7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:25:04.036698 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d691e5-713c-4030-bbc6-2ff82a64fed7 admin admin] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:25:04.037011 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d691e5-713c-4030-bbc6-2ff82a64fed7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:25:04.037278 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d691e5-713c-4030-bbc6-2ff82a64fed7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:25:04.037911 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d691e5-713c-4030-bbc6-2ff82a64fed7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:25:04.039830 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d691e5-713c-4030-bbc6-2ff82a64fed7 admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:25:04.054759 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=94,keystone:DELETE=4,keystone:INSERT=2 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:25:04.073064 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 997/1971] 10.222.0.15 () {60 vars in 1153 bytes} [Thu Sep 15 12:25:03 2022] POST /identity/v3/projects => generated 383 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 12:25:04.077840 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f85a6d-0f45-4520-848a-45e81821d856 None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:25:04.089121 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f85a6d-0f45-4520-848a-45e81821d856 None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:25:04.125842 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:25:04.153670 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d3457b-4cca-4db8-add2-f26aab8fc51d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:25:04.154359 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d3457b-4cca-4db8-add2-f26aab8fc51d admin admin] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:25:04.154549 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d3457b-4cca-4db8-add2-f26aab8fc51d admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:25:04.154843 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d3457b-4cca-4db8-add2-f26aab8fc51d admin admin] PATH_INFO: `/v3/users` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:25:04.155470 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d3457b-4cca-4db8-add2-f26aab8fc51d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:25:04.157448 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d3457b-4cca-4db8-add2-f26aab8fc51d admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:25:04.183969 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 975/1972] 10.222.0.15 () {60 vars in 1144 bytes} [Thu Sep 15 12:25:04 2022] POST /identity/v3/users => generated 298 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 12:25:04.188033 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d691e5-713c-4030-bbc6-2ff82a64fed7 admin admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:25:04.199190 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d691e5-713c-4030-bbc6-2ff82a64fed7 admin admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:25:04.203873 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20a0684-f322-420d-b777-79a6b98ba33d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:25:04.204465 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20a0684-f322-420d-b777-79a6b98ba33d admin admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:25:04.204728 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20a0684-f322-420d-b777-79a6b98ba33d admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:25:04.205066 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20a0684-f322-420d-b777-79a6b98ba33d admin admin] PATH_INFO: `/v3/roles` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:25:04.205711 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d20a0684-f322-420d-b777-79a6b98ba33d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:25:04.207383 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d20a0684-f322-420d-b777-79a6b98ba33d admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:25:04.221544 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 998/1973] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 12:25:04 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 15 12:25:04.225928 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d3457b-4cca-4db8-add2-f26aab8fc51d admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:25:04.236523 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d3457b-4cca-4db8-add2-f26aab8fc51d admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:25:04.241501 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2b4e5f-feea-42da-9cba-ecc512c952d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:25:04.242750 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2b4e5f-feea-42da-9cba-ecc512c952d9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:25:04.243024 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2b4e5f-feea-42da-9cba-ecc512c952d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:25:04.243291 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2b4e5f-feea-42da-9cba-ecc512c952d9 admin admin] PATH_INFO: `/v3/projects/3c486beb2a854b8b8c37bcd768e651e8/users/2b59c1b53b7a47c5b2c2f084ef11f130/roles/9c769b5c82884766b2c510b1f0f1364c` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:25:04.259721 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2b4e5f-feea-42da-9cba-ecc512c952d9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3c486beb2a854b8b8c37bcd768e651e8, user_id=2b59c1b53b7a47c5b2c2f084ef11f130, role_id=9c769b5c82884766b2c510b1f0f1364c)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:25:04.262448 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2b4e5f-feea-42da-9cba-ecc512c952d9 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:25:04.276438 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 976/1974] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 15 12:25:04 2022] PUT /identity/v3/projects/3c486beb2a854b8b8c37bcd768e651e8/users/2b59c1b53b7a47c5b2c2f084ef11f130/roles/9c769b5c82884766b2c510b1f0f1364c => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:25:04.280479 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20a0684-f322-420d-b777-79a6b98ba33d admin admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:25:04.290798 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d20a0684-f322-420d-b777-79a6b98ba33d admin admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:25:04.358378 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f15394d-fce5-42c0-a5af-5ecc3c59efa7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:25:04.358996 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f15394d-fce5-42c0-a5af-5ecc3c59efa7 admin admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:25:04.359278 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f15394d-fce5-42c0-a5af-5ecc3c59efa7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:25:04.359548 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f15394d-fce5-42c0-a5af-5ecc3c59efa7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:25:04.360378 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f15394d-fce5-42c0-a5af-5ecc3c59efa7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:25:04.361959 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f15394d-fce5-42c0-a5af-5ecc3c59efa7 admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:25:04.372659 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 999/1975] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 12:25:04 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 12:25:04.376914 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2b4e5f-feea-42da-9cba-ecc512c952d9 admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:25:04.388742 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2b4e5f-feea-42da-9cba-ecc512c952d9 admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:25:04.393913 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57fecf1a-92e2-482c-9592-20bc72e6faad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:25:04.394314 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fecf1a-92e2-482c-9592-20bc72e6faad admin admin] REQUEST_METHOD: `PUT` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:25:04.394799 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fecf1a-92e2-482c-9592-20bc72e6faad admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:25:04.395080 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fecf1a-92e2-482c-9592-20bc72e6faad admin admin] PATH_INFO: `/v3/projects/3c486beb2a854b8b8c37bcd768e651e8/users/2b59c1b53b7a47c5b2c2f084ef11f130/roles/9c769b5c82884766b2c510b1f0f1364c` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:25:04.397134 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57fecf1a-92e2-482c-9592-20bc72e6faad admin admin] RBAC: Authorizing `identity:create_grant(project_id=3c486beb2a854b8b8c37bcd768e651e8, user_id=2b59c1b53b7a47c5b2c2f084ef11f130, role_id=9c769b5c82884766b2c510b1f0f1364c)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:25:04.400452 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57fecf1a-92e2-482c-9592-20bc72e6faad admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:25:04.412838 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 977/1976] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 15 12:25:04 2022] PUT /identity/v3/projects/3c486beb2a854b8b8c37bcd768e651e8/users/2b59c1b53b7a47c5b2c2f084ef11f130/roles/9c769b5c82884766b2c510b1f0f1364c => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:25:04.422526 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b7eef8-c377-4921-8290-85dd79f68652 None None] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:25:04.422814 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b7eef8-c377-4921-8290-85dd79f68652 None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:25:04.423114 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b7eef8-c377-4921-8290-85dd79f68652 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:25:04.462616 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.auth.core [None req-14b7eef8-c377-4921-8290-85dd79f68652 None None] MFA Rules not processed for user `2b59c1b53b7a47c5b2c2f084ef11f130`. Rule list: `[]` (Enabled: `True`). {{(pid=81449) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:25:04.463811 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-14b7eef8-c377-4921-8290-85dd79f68652 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:25:04.539077 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1000/1977] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 12:25:04 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:25:04.550501 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57fecf1a-92e2-482c-9592-20bc72e6faad admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:25:04.552145 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-57fecf1a-92e2-482c-9592-20bc72e6faad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:25:04.607685 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57fecf1a-92e2-482c-9592-20bc72e6faad admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:25:04.611246 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32fb01f1-49f5-4f78-b0ab-5f7e44ea1547 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8191d3ccb874529b32f8d0026db629d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '540a32ef88764e8c8d9201ae22f6ddaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:25:04.611848 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fb01f1-49f5-4f78-b0ab-5f7e44ea1547 service cinder] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:25:04.612125 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fb01f1-49f5-4f78-b0ab-5f7e44ea1547 service cinder] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:25:04.612391 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fb01f1-49f5-4f78-b0ab-5f7e44ea1547 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:25:04.672799 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32fb01f1-49f5-4f78-b0ab-5f7e44ea1547 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:25:04.674228 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32fb01f1-49f5-4f78-b0ab-5f7e44ea1547 service cinder] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:25:04.683411 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 978/1978] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 12:25:04 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 12:25:07.642528 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b7eef8-c377-4921-8290-85dd79f68652 None None] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:25:07.644106 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-14b7eef8-c377-4921-8290-85dd79f68652 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:25:07.721200 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b7eef8-c377-4921-8290-85dd79f68652 None None] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:25:07.725772 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695ea528-9e02-459a-8d48-db7ee0aaccc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:25:07.726352 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695ea528-9e02-459a-8d48-db7ee0aaccc7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:25:07.726631 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695ea528-9e02-459a-8d48-db7ee0aaccc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:25:07.726901 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695ea528-9e02-459a-8d48-db7ee0aaccc7 admin admin] PATH_INFO: `/v3/users/7f894b889a624fb799caec1fd2559203` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:25:07.728141 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695ea528-9e02-459a-8d48-db7ee0aaccc7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7f894b889a624fb799caec1fd2559203)` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:25:07.730065 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695ea528-9e02-459a-8d48-db7ee0aaccc7 admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:25:07.734968 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:25:07.784190 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-695ea528-9e02-459a-8d48-db7ee0aaccc7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7f894b889a624fb799caec1fd2559203'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:25:07.792669 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-695ea528-9e02-459a-8d48-db7ee0aaccc7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7f894b889a624fb799caec1fd2559203'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:25:07.807814 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-695ea528-9e02-459a-8d48-db7ee0aaccc7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7f894b889a624fb799caec1fd2559203'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:25:07.809317 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-695ea528-9e02-459a-8d48-db7ee0aaccc7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7f894b889a624fb799caec1fd2559203'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:25:07.817310 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-695ea528-9e02-459a-8d48-db7ee0aaccc7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7f894b889a624fb799caec1fd2559203'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:25:07.824686 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1001/1979] 10.222.0.15 () {58 vars in 1224 bytes} [Thu Sep 15 12:25:07 2022] DELETE /identity/v3/users/7f894b889a624fb799caec1fd2559203 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:25:07.828851 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b760fb-e5dc-46ee-bf10-79c30da33d46 None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:25:07.829333 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b760fb-e5dc-46ee-bf10-79c30da33d46 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:25:07.829586 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b760fb-e5dc-46ee-bf10-79c30da33d46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:25:07.850615 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-f4b760fb-e5dc-46ee-bf10-79c30da33d46 None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:25:07.852183 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-f4b760fb-e5dc-46ee-bf10-79c30da33d46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:25:07.897457 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:25:07.921531 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 979/1980] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 12:25:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:25:07.933224 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695ea528-9e02-459a-8d48-db7ee0aaccc7 admin admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:25:07.934822 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-695ea528-9e02-459a-8d48-db7ee0aaccc7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:25:08.023303 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695ea528-9e02-459a-8d48-db7ee0aaccc7 admin admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:25:08.026997 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421d44a2-b68e-499b-9a0c-13f2013e22bc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfbb4d6809064f9cb9e06626ed7d6814', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '540a32ef88764e8c8d9201ae22f6ddaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:25:08.027611 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421d44a2-b68e-499b-9a0c-13f2013e22bc service neutron] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:25:08.027891 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421d44a2-b68e-499b-9a0c-13f2013e22bc service neutron] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:25:08.028161 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421d44a2-b68e-499b-9a0c-13f2013e22bc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:25:08.038808 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421d44a2-b68e-499b-9a0c-13f2013e22bc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:25:08.040413 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421d44a2-b68e-499b-9a0c-13f2013e22bc service neutron] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:25:08.050643 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1002/1981] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 12:25:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 12:25:08.470814 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b760fb-e5dc-46ee-bf10-79c30da33d46 None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:25:08.472423 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-f4b760fb-e5dc-46ee-bf10-79c30da33d46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:25:08.486502 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b760fb-e5dc-46ee-bf10-79c30da33d46 None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:25:08.491606 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551b02e1-3d1e-4a3e-abfb-35e8b44d1a04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:25:08.492355 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551b02e1-3d1e-4a3e-abfb-35e8b44d1a04 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:25:08.492637 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551b02e1-3d1e-4a3e-abfb-35e8b44d1a04 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:25:08.492969 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551b02e1-3d1e-4a3e-abfb-35e8b44d1a04 admin admin] PATH_INFO: `/v3/projects/6dab91f787bf49198b77b83c5468869f` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:25:08.494728 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551b02e1-3d1e-4a3e-abfb-35e8b44d1a04 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6dab91f787bf49198b77b83c5468869f)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:25:08.496836 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551b02e1-3d1e-4a3e-abfb-35e8b44d1a04 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:25:08.543488 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-551b02e1-3d1e-4a3e-abfb-35e8b44d1a04 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6dab91f787bf49198b77b83c5468869f', 'request_id': 'req-551b02e1-3d1e-4a3e-abfb-35e8b44d1a04'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:25:08.550625 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-551b02e1-3d1e-4a3e-abfb-35e8b44d1a04 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6dab91f787bf49198b77b83c5468869f', 'request_id': 'req-551b02e1-3d1e-4a3e-abfb-35e8b44d1a04'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:25:08.557006 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-551b02e1-3d1e-4a3e-abfb-35e8b44d1a04 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6dab91f787bf49198b77b83c5468869f', 'request_id': 'req-551b02e1-3d1e-4a3e-abfb-35e8b44d1a04'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:25:08.571271 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-551b02e1-3d1e-4a3e-abfb-35e8b44d1a04 admin admin] The token cache is being invalidate because project 6dab91f787bf49198b77b83c5468869f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81448) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 12:25:08.571639 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-551b02e1-3d1e-4a3e-abfb-35e8b44d1a04 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:25:08.572302 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-551b02e1-3d1e-4a3e-abfb-35e8b44d1a04 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:25:08.573823 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 980/1982] 10.222.0.15 () {58 vars in 1233 bytes} [Thu Sep 15 12:25:08 2022] DELETE /identity/v3/projects/6dab91f787bf49198b77b83c5468869f => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 12:25:08.577846 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421d44a2-b68e-499b-9a0c-13f2013e22bc service neutron] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:25:08.587966 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421d44a2-b68e-499b-9a0c-13f2013e22bc service neutron] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:25:08.648325 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae878b40-822d-48f9-85e6-c4d4e7eff6dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:25:08.649644 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae878b40-822d-48f9-85e6-c4d4e7eff6dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:25:08.649957 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae878b40-822d-48f9-85e6-c4d4e7eff6dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:25:08.650248 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae878b40-822d-48f9-85e6-c4d4e7eff6dc admin admin] PATH_INFO: `/v3/users/c27b01fc280d4a8486fe48b1db159935` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:25:08.651911 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae878b40-822d-48f9-85e6-c4d4e7eff6dc admin admin] RBAC: Authorizing `identity:delete_user(user_id=c27b01fc280d4a8486fe48b1db159935)` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:25:08.655009 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae878b40-822d-48f9-85e6-c4d4e7eff6dc admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:25:08.694976 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:25:08.717017 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-ae878b40-822d-48f9-85e6-c4d4e7eff6dc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c27b01fc280d4a8486fe48b1db159935'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:25:08.725244 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-ae878b40-822d-48f9-85e6-c4d4e7eff6dc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c27b01fc280d4a8486fe48b1db159935'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:25:08.738029 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-ae878b40-822d-48f9-85e6-c4d4e7eff6dc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c27b01fc280d4a8486fe48b1db159935'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:25:08.738696 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-ae878b40-822d-48f9-85e6-c4d4e7eff6dc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c27b01fc280d4a8486fe48b1db159935'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:25:08.746394 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-ae878b40-822d-48f9-85e6-c4d4e7eff6dc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c27b01fc280d4a8486fe48b1db159935'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:25:08.752456 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1003/1983] 10.222.0.15 () {58 vars in 1224 bytes} [Thu Sep 15 12:25:08 2022] DELETE /identity/v3/users/c27b01fc280d4a8486fe48b1db159935 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:25:09.091567 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551b02e1-3d1e-4a3e-abfb-35e8b44d1a04 admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:25:09.093715 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-551b02e1-3d1e-4a3e-abfb-35e8b44d1a04 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:25:09.109983 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551b02e1-3d1e-4a3e-abfb-35e8b44d1a04 admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:25:09.167483 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e8e7f4-5ab7-4ce2-8c5b-2e8537bf33ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:25:09.168206 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e8e7f4-5ab7-4ce2-8c5b-2e8537bf33ef admin admin] REQUEST_METHOD: `DELETE` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:25:09.168497 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e8e7f4-5ab7-4ce2-8c5b-2e8537bf33ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:25:09.168788 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e8e7f4-5ab7-4ce2-8c5b-2e8537bf33ef admin admin] PATH_INFO: `/v3/projects/64fc22464ba0487c83257064001b9d71` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:25:09.169992 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e8e7f4-5ab7-4ce2-8c5b-2e8537bf33ef admin admin] RBAC: Authorizing `identity:delete_project(project_id=64fc22464ba0487c83257064001b9d71)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:25:09.172191 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e8e7f4-5ab7-4ce2-8c5b-2e8537bf33ef admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:25:09.223486 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-d7e8e7f4-5ab7-4ce2-8c5b-2e8537bf33ef admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '64fc22464ba0487c83257064001b9d71', 'request_id': 'req-d7e8e7f4-5ab7-4ce2-8c5b-2e8537bf33ef'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:25:09.231991 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-d7e8e7f4-5ab7-4ce2-8c5b-2e8537bf33ef admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '64fc22464ba0487c83257064001b9d71', 'request_id': 'req-d7e8e7f4-5ab7-4ce2-8c5b-2e8537bf33ef'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:25:09.237939 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-d7e8e7f4-5ab7-4ce2-8c5b-2e8537bf33ef admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '64fc22464ba0487c83257064001b9d71', 'request_id': 'req-d7e8e7f4-5ab7-4ce2-8c5b-2e8537bf33ef'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:25:09.240280 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:25:09.252287 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-d7e8e7f4-5ab7-4ce2-8c5b-2e8537bf33ef admin admin] The token cache is being invalidate because project 64fc22464ba0487c83257064001b9d71 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81448) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 12:25:09.252729 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-d7e8e7f4-5ab7-4ce2-8c5b-2e8537bf33ef admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:25:09.253572 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-d7e8e7f4-5ab7-4ce2-8c5b-2e8537bf33ef admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:25:09.255167 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 981/1984] 10.222.0.15 () {58 vars in 1233 bytes} [Thu Sep 15 12:25:09 2022] DELETE /identity/v3/projects/64fc22464ba0487c83257064001b9d71 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:25:09.259343 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae878b40-822d-48f9-85e6-c4d4e7eff6dc admin admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:25:09.261061 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-ae878b40-822d-48f9-85e6-c4d4e7eff6dc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:25:09.275096 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae878b40-822d-48f9-85e6-c4d4e7eff6dc admin admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:25:09.341943 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c291ccd1-2f44-48b4-b777-9365b7ce6b8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:25:09.342861 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c291ccd1-2f44-48b4-b777-9365b7ce6b8d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:25:09.343331 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c291ccd1-2f44-48b4-b777-9365b7ce6b8d admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:25:09.343607 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c291ccd1-2f44-48b4-b777-9365b7ce6b8d admin admin] PATH_INFO: `/v3/users/702a92cca62b4219a6cfc86d766be78e` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:25:09.344938 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c291ccd1-2f44-48b4-b777-9365b7ce6b8d admin admin] RBAC: Authorizing `identity:delete_user(user_id=702a92cca62b4219a6cfc86d766be78e)` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:25:09.347940 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c291ccd1-2f44-48b4-b777-9365b7ce6b8d admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:25:09.402375 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-c291ccd1-2f44-48b4-b777-9365b7ce6b8d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '702a92cca62b4219a6cfc86d766be78e'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:25:09.413511 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-c291ccd1-2f44-48b4-b777-9365b7ce6b8d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '702a92cca62b4219a6cfc86d766be78e'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:25:09.431699 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-c291ccd1-2f44-48b4-b777-9365b7ce6b8d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '702a92cca62b4219a6cfc86d766be78e'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:25:09.432596 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-c291ccd1-2f44-48b4-b777-9365b7ce6b8d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '702a92cca62b4219a6cfc86d766be78e'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:25:09.442841 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-c291ccd1-2f44-48b4-b777-9365b7ce6b8d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '702a92cca62b4219a6cfc86d766be78e'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:25:09.449629 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1004/1985] 10.222.0.15 () {58 vars in 1224 bytes} [Thu Sep 15 12:25:09 2022] DELETE /identity/v3/users/702a92cca62b4219a6cfc86d766be78e => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:25:10.174135 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e8e7f4-5ab7-4ce2-8c5b-2e8537bf33ef admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:25:10.175620 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-d7e8e7f4-5ab7-4ce2-8c5b-2e8537bf33ef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:25:10.196326 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c291ccd1-2f44-48b4-b777-9365b7ce6b8d admin admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:25:10.197652 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e8e7f4-5ab7-4ce2-8c5b-2e8537bf33ef admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:25:10.198687 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-c291ccd1-2f44-48b4-b777-9365b7ce6b8d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:25:10.235014 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:25:10.270463 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc08f37-eba9-4aa6-a846-f789bad7e2f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:25:10.271210 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc08f37-eba9-4aa6-a846-f789bad7e2f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:25:10.271724 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc08f37-eba9-4aa6-a846-f789bad7e2f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:25:10.272190 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc08f37-eba9-4aa6-a846-f789bad7e2f9 admin admin] PATH_INFO: `/v3/projects/7156351a003c4b9fa185fc09144746cd` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:25:10.273427 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc08f37-eba9-4aa6-a846-f789bad7e2f9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7156351a003c4b9fa185fc09144746cd)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:25:10.275941 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc08f37-eba9-4aa6-a846-f789bad7e2f9 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:25:10.284913 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c291ccd1-2f44-48b4-b777-9365b7ce6b8d admin admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:25:10.289414 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939d7ea9-c921-4065-a0b9-4af516b24746 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8191d3ccb874529b32f8d0026db629d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '540a32ef88764e8c8d9201ae22f6ddaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:25:10.290810 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939d7ea9-c921-4065-a0b9-4af516b24746 service cinder] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:25:10.291158 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939d7ea9-c921-4065-a0b9-4af516b24746 service cinder] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:25:10.291433 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939d7ea9-c921-4065-a0b9-4af516b24746 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:25:10.293334 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-939d7ea9-c921-4065-a0b9-4af516b24746 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:25:10.327560 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-cbc08f37-eba9-4aa6-a846-f789bad7e2f9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7156351a003c4b9fa185fc09144746cd', 'request_id': 'req-cbc08f37-eba9-4aa6-a846-f789bad7e2f9'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:25:10.338955 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-cbc08f37-eba9-4aa6-a846-f789bad7e2f9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7156351a003c4b9fa185fc09144746cd', 'request_id': 'req-cbc08f37-eba9-4aa6-a846-f789bad7e2f9'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:25:10.350529 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-cbc08f37-eba9-4aa6-a846-f789bad7e2f9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7156351a003c4b9fa185fc09144746cd', 'request_id': 'req-cbc08f37-eba9-4aa6-a846-f789bad7e2f9'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:25:10.362542 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-cbc08f37-eba9-4aa6-a846-f789bad7e2f9 admin admin] The token cache is being invalidate because project 7156351a003c4b9fa185fc09144746cd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81448) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 12:25:10.363462 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-cbc08f37-eba9-4aa6-a846-f789bad7e2f9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:25:10.364400 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-cbc08f37-eba9-4aa6-a846-f789bad7e2f9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:25:10.365939 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 982/1986] 10.222.0.15 () {58 vars in 1233 bytes} [Thu Sep 15 12:25:10 2022] DELETE /identity/v3/projects/7156351a003c4b9fa185fc09144746cd => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 12:25:10.370903 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939d7ea9-c921-4065-a0b9-4af516b24746 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:25:10.372353 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ee5795-09dc-4068-afef-eafd46eae670 None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:25:10.372504 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939d7ea9-c921-4065-a0b9-4af516b24746 service cinder] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:25:10.372858 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ee5795-09dc-4068-afef-eafd46eae670 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:25:10.373248 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ee5795-09dc-4068-afef-eafd46eae670 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:25:10.383579 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1005/1987] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 12:25:10 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 12:25:10.394357 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-26ee5795-09dc-4068-afef-eafd46eae670 None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:25:10.395714 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-26ee5795-09dc-4068-afef-eafd46eae670 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:25:10.469584 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 983/1988] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 12:25:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 12:25:10.480448 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817217ac-4432-467f-85b0-9eceb38afa5c None None] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:25:10.480997 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817217ac-4432-467f-85b0-9eceb38afa5c None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:25:10.481910 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817217ac-4432-467f-85b0-9eceb38afa5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:25:10.507247 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.auth.core [None req-817217ac-4432-467f-85b0-9eceb38afa5c None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81449) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:25:10.509692 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-817217ac-4432-467f-85b0-9eceb38afa5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:25:10.518278 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1006/1989] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 12:25:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:25:10.522536 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ee5795-09dc-4068-afef-eafd46eae670 None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:25:10.537338 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ee5795-09dc-4068-afef-eafd46eae670 None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:25:10.543326 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1fd84b-d686-4832-bb08-5931b1e13338 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:25:10.543976 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1fd84b-d686-4832-bb08-5931b1e13338 admin admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:25:10.544258 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1fd84b-d686-4832-bb08-5931b1e13338 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:25:10.544531 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1fd84b-d686-4832-bb08-5931b1e13338 admin admin] PATH_INFO: `/v3/domains` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:25:10.545336 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1fd84b-d686-4832-bb08-5931b1e13338 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:25:10.545521 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1fd84b-d686-4832-bb08-5931b1e13338 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:25:10.547198 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1fd84b-d686-4832-bb08-5931b1e13338 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:25:10.562061 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 984/1990] 10.222.0.15 () {58 vars in 1153 bytes} [Thu Sep 15 12:25:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:25:10.566434 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817217ac-4432-467f-85b0-9eceb38afa5c None None] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:25:10.581609 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817217ac-4432-467f-85b0-9eceb38afa5c None None] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:25:10.587795 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae06f45c-8950-40c8-b71a-528a54c7f848 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:25:10.588466 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae06f45c-8950-40c8-b71a-528a54c7f848 admin admin] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:25:10.588746 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae06f45c-8950-40c8-b71a-528a54c7f848 admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:25:10.589042 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae06f45c-8950-40c8-b71a-528a54c7f848 admin admin] PATH_INFO: `/v3/projects` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:25:10.589739 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae06f45c-8950-40c8-b71a-528a54c7f848 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:25:10.591863 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae06f45c-8950-40c8-b71a-528a54c7f848 admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:25:10.626541 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1007/1991] 10.222.0.15 () {60 vars in 1153 bytes} [Thu Sep 15 12:25:10 2022] POST /identity/v3/projects => generated 387 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 12:25:10.630621 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1fd84b-d686-4832-bb08-5931b1e13338 admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:25:10.643921 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1fd84b-d686-4832-bb08-5931b1e13338 admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:25:10.658448 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:25:10.739968 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcaf392f-f93f-4254-bc68-81bb052a1762 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:25:10.740982 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcaf392f-f93f-4254-bc68-81bb052a1762 admin admin] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:25:10.741596 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcaf392f-f93f-4254-bc68-81bb052a1762 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:25:10.741871 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcaf392f-f93f-4254-bc68-81bb052a1762 admin admin] PATH_INFO: `/v3/users` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:25:10.742897 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcaf392f-f93f-4254-bc68-81bb052a1762 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:25:10.746407 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcaf392f-f93f-4254-bc68-81bb052a1762 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:25:10.782095 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 985/1992] 10.222.0.15 () {60 vars in 1144 bytes} [Thu Sep 15 12:25:10 2022] POST /identity/v3/users => generated 294 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 15 12:25:10.786298 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae06f45c-8950-40c8-b71a-528a54c7f848 admin admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:25:10.809926 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae06f45c-8950-40c8-b71a-528a54c7f848 admin admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:25:10.821475 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287a767d-5712-462c-b65d-8b380b4bc239 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:25:10.822187 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287a767d-5712-462c-b65d-8b380b4bc239 admin admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:25:10.822531 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287a767d-5712-462c-b65d-8b380b4bc239 admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:25:10.822874 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-287a767d-5712-462c-b65d-8b380b4bc239 admin admin] PATH_INFO: `/v3/roles` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:25:10.823590 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287a767d-5712-462c-b65d-8b380b4bc239 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:25:10.825351 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-287a767d-5712-462c-b65d-8b380b4bc239 admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:25:10.851095 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1008/1993] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 12:25:10 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 12:25:10.852573 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcaf392f-f93f-4254-bc68-81bb052a1762 admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:25:10.871068 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcaf392f-f93f-4254-bc68-81bb052a1762 admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:25:10.876541 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ab2e78-f547-4e73-92b8-80227d3303ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:25:10.877472 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ab2e78-f547-4e73-92b8-80227d3303ae admin admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:25:10.878021 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ab2e78-f547-4e73-92b8-80227d3303ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:25:10.878368 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ab2e78-f547-4e73-92b8-80227d3303ae admin admin] PATH_INFO: `/v3/roles` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:25:10.880069 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ab2e78-f547-4e73-92b8-80227d3303ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:25:10.882262 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ab2e78-f547-4e73-92b8-80227d3303ae admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:25:10.895972 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 986/1994] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 12:25:10 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 12:25:10.900423 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287a767d-5712-462c-b65d-8b380b4bc239 admin admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:25:10.923913 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-287a767d-5712-462c-b65d-8b380b4bc239 admin admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:25:10.931435 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d1cad7-aac7-46b4-b76d-3f0d6ec411c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:25:10.932386 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d1cad7-aac7-46b4-b76d-3f0d6ec411c2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:25:10.932807 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d1cad7-aac7-46b4-b76d-3f0d6ec411c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:25:10.933245 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d1cad7-aac7-46b4-b76d-3f0d6ec411c2 admin admin] PATH_INFO: `/v3/projects/4c5c7aeb70f148ad83169665e6fa6f32/users/329d65c2d27c4f9fb13d417e4e57c33f/roles/0679d6628a25438d90dc4bdc0401eda2` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:25:10.979638 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d1cad7-aac7-46b4-b76d-3f0d6ec411c2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4c5c7aeb70f148ad83169665e6fa6f32, user_id=329d65c2d27c4f9fb13d417e4e57c33f, role_id=0679d6628a25438d90dc4bdc0401eda2)` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:25:10.983989 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d1cad7-aac7-46b4-b76d-3f0d6ec411c2 admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:25:11.006597 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1009/1995] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 15 12:25:10 2022] PUT /identity/v3/projects/4c5c7aeb70f148ad83169665e6fa6f32/users/329d65c2d27c4f9fb13d417e4e57c33f/roles/0679d6628a25438d90dc4bdc0401eda2 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:25:11.019167 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332b8f43-2ec3-4f1d-82c1-2bf9f6d6f3db None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:25:11.019367 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332b8f43-2ec3-4f1d-82c1-2bf9f6d6f3db None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:25:11.019554 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332b8f43-2ec3-4f1d-82c1-2bf9f6d6f3db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:25:11.080983 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-332b8f43-2ec3-4f1d-82c1-2bf9f6d6f3db None None] MFA Rules not processed for user `329d65c2d27c4f9fb13d417e4e57c33f`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:25:11.082302 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-332b8f43-2ec3-4f1d-82c1-2bf9f6d6f3db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:25:11.199316 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 987/1996] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 12:25:11 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:25:11.204685 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d1cad7-aac7-46b4-b76d-3f0d6ec411c2 admin admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:25:11.309869 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d1cad7-aac7-46b4-b76d-3f0d6ec411c2 admin admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:25:11.314987 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b68f6f5-e360-469d-a9a3-8b9bdfe6cfa8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:25:11.315707 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b68f6f5-e360-469d-a9a3-8b9bdfe6cfa8 admin admin] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:25:11.316059 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b68f6f5-e360-469d-a9a3-8b9bdfe6cfa8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:25:11.316439 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b68f6f5-e360-469d-a9a3-8b9bdfe6cfa8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:25:11.317189 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b68f6f5-e360-469d-a9a3-8b9bdfe6cfa8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:25:11.319207 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b68f6f5-e360-469d-a9a3-8b9bdfe6cfa8 admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:25:11.339747 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:25:11.357956 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1010/1997] 10.222.0.15 () {60 vars in 1153 bytes} [Thu Sep 15 12:25:11 2022] POST /identity/v3/projects => generated 385 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 12:25:11.361974 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332b8f43-2ec3-4f1d-82c1-2bf9f6d6f3db None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:25:11.373511 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332b8f43-2ec3-4f1d-82c1-2bf9f6d6f3db None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:25:11.435398 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15ed025-fb94-479e-adc1-92081d7c1baf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:25:11.436119 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15ed025-fb94-479e-adc1-92081d7c1baf admin admin] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:25:11.436409 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15ed025-fb94-479e-adc1-92081d7c1baf admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:25:11.436684 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15ed025-fb94-479e-adc1-92081d7c1baf admin admin] PATH_INFO: `/v3/users` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:25:11.437398 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15ed025-fb94-479e-adc1-92081d7c1baf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:25:11.439318 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15ed025-fb94-479e-adc1-92081d7c1baf admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:25:11.463575 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 988/1998] 10.222.0.15 () {60 vars in 1144 bytes} [Thu Sep 15 12:25:11 2022] POST /identity/v3/users => generated 299 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 12:25:11.467729 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b68f6f5-e360-469d-a9a3-8b9bdfe6cfa8 admin admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:25:11.480154 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b68f6f5-e360-469d-a9a3-8b9bdfe6cfa8 admin admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:25:11.484938 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82551c00-682d-4136-b493-2b2f05396af6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:25:11.485550 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82551c00-682d-4136-b493-2b2f05396af6 admin admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:25:11.485824 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82551c00-682d-4136-b493-2b2f05396af6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:25:11.486090 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82551c00-682d-4136-b493-2b2f05396af6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:25:11.486906 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82551c00-682d-4136-b493-2b2f05396af6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:25:11.488490 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82551c00-682d-4136-b493-2b2f05396af6 admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:25:11.505254 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1011/1999] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 12:25:11 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 12:25:11.506432 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15ed025-fb94-479e-adc1-92081d7c1baf admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:25:11.518083 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15ed025-fb94-479e-adc1-92081d7c1baf admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:25:11.523126 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0264650-a104-43a6-97be-89c7e519b6cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:25:11.524065 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0264650-a104-43a6-97be-89c7e519b6cf admin admin] REQUEST_METHOD: `PUT` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:25:11.524518 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0264650-a104-43a6-97be-89c7e519b6cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:25:11.524830 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0264650-a104-43a6-97be-89c7e519b6cf admin admin] PATH_INFO: `/v3/projects/140d281723af41beb956c46aecbafe6a/users/8ea8dce4c7b24a06b27b96f8089130aa/roles/9c769b5c82884766b2c510b1f0f1364c` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:25:11.534303 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:25:11.543769 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0264650-a104-43a6-97be-89c7e519b6cf admin admin] RBAC: Authorizing `identity:create_grant(project_id=140d281723af41beb956c46aecbafe6a, user_id=8ea8dce4c7b24a06b27b96f8089130aa, role_id=9c769b5c82884766b2c510b1f0f1364c)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:25:11.546852 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0264650-a104-43a6-97be-89c7e519b6cf admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:25:11.562533 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 989/2000] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 15 12:25:11 2022] PUT /identity/v3/projects/140d281723af41beb956c46aecbafe6a/users/8ea8dce4c7b24a06b27b96f8089130aa/roles/9c769b5c82884766b2c510b1f0f1364c => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:25:11.566437 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82551c00-682d-4136-b493-2b2f05396af6 admin admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:25:11.580543 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82551c00-682d-4136-b493-2b2f05396af6 admin admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:25:11.656145 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbddcde-7eb1-4c86-970d-27e39232c2bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:25:11.656790 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbddcde-7eb1-4c86-970d-27e39232c2bf admin admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:25:11.657489 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbddcde-7eb1-4c86-970d-27e39232c2bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:25:11.657489 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbddcde-7eb1-4c86-970d-27e39232c2bf admin admin] PATH_INFO: `/v3/roles` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:25:11.658028 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dbddcde-7eb1-4c86-970d-27e39232c2bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:25:11.659822 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dbddcde-7eb1-4c86-970d-27e39232c2bf admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:25:11.672035 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1012/2001] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 12:25:11 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 12:25:11.676406 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0264650-a104-43a6-97be-89c7e519b6cf admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:25:11.686762 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0264650-a104-43a6-97be-89c7e519b6cf admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:25:11.692119 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089ac859-12b1-4994-bd19-a9c2868f630d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:25:11.693000 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089ac859-12b1-4994-bd19-a9c2868f630d admin admin] REQUEST_METHOD: `PUT` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:25:11.693165 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089ac859-12b1-4994-bd19-a9c2868f630d admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:25:11.693730 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089ac859-12b1-4994-bd19-a9c2868f630d admin admin] PATH_INFO: `/v3/projects/140d281723af41beb956c46aecbafe6a/users/8ea8dce4c7b24a06b27b96f8089130aa/roles/9c769b5c82884766b2c510b1f0f1364c` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:25:11.695727 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089ac859-12b1-4994-bd19-a9c2868f630d admin admin] RBAC: Authorizing `identity:create_grant(project_id=140d281723af41beb956c46aecbafe6a, user_id=8ea8dce4c7b24a06b27b96f8089130aa, role_id=9c769b5c82884766b2c510b1f0f1364c)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:25:11.698587 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089ac859-12b1-4994-bd19-a9c2868f630d admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:25:11.709940 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 990/2002] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 15 12:25:11 2022] PUT /identity/v3/projects/140d281723af41beb956c46aecbafe6a/users/8ea8dce4c7b24a06b27b96f8089130aa/roles/9c769b5c82884766b2c510b1f0f1364c => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:25:11.723240 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b875b99-9dbb-41fd-9a6c-5790e674fa0c None None] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:25:11.723793 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b875b99-9dbb-41fd-9a6c-5790e674fa0c None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:25:11.724095 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b875b99-9dbb-41fd-9a6c-5790e674fa0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:25:11.756769 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.auth.core [None req-4b875b99-9dbb-41fd-9a6c-5790e674fa0c None None] MFA Rules not processed for user `8ea8dce4c7b24a06b27b96f8089130aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81449) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:25:11.757998 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-4b875b99-9dbb-41fd-9a6c-5790e674fa0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:25:11.837806 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1013/2003] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 12:25:11 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:25:11.846133 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089ac859-12b1-4994-bd19-a9c2868f630d admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:25:11.847771 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-089ac859-12b1-4994-bd19-a9c2868f630d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:25:11.920090 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089ac859-12b1-4994-bd19-a9c2868f630d admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:25:11.923704 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe9d57b-e9c7-43d8-be72-a4508d436e0e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8191d3ccb874529b32f8d0026db629d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '540a32ef88764e8c8d9201ae22f6ddaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:25:11.925140 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe9d57b-e9c7-43d8-be72-a4508d436e0e service cinder] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:25:11.925606 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe9d57b-e9c7-43d8-be72-a4508d436e0e service cinder] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:25:11.926055 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe9d57b-e9c7-43d8-be72-a4508d436e0e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:25:11.937289 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe9d57b-e9c7-43d8-be72-a4508d436e0e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:25:11.938928 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe9d57b-e9c7-43d8-be72-a4508d436e0e service cinder] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:25:11.950721 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 991/2004] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 12:25:11 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 12:25:12.675123 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b875b99-9dbb-41fd-9a6c-5790e674fa0c None None] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:25:12.756999 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b875b99-9dbb-41fd-9a6c-5790e674fa0c None None] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:25:12.762894 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53cb132-94a2-44e8-b9d5-659a33fee2e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:25:12.763555 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53cb132-94a2-44e8-b9d5-659a33fee2e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:25:12.764194 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53cb132-94a2-44e8-b9d5-659a33fee2e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:25:12.764470 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53cb132-94a2-44e8-b9d5-659a33fee2e6 admin admin] PATH_INFO: `/v3/users/329d65c2d27c4f9fb13d417e4e57c33f` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:25:12.765656 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53cb132-94a2-44e8-b9d5-659a33fee2e6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=329d65c2d27c4f9fb13d417e4e57c33f)` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:25:12.767607 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53cb132-94a2-44e8-b9d5-659a33fee2e6 admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:25:12.773971 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:25:12.816457 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-f53cb132-94a2-44e8-b9d5-659a33fee2e6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '329d65c2d27c4f9fb13d417e4e57c33f'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:25:12.824124 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-f53cb132-94a2-44e8-b9d5-659a33fee2e6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '329d65c2d27c4f9fb13d417e4e57c33f'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:25:12.837154 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-f53cb132-94a2-44e8-b9d5-659a33fee2e6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '329d65c2d27c4f9fb13d417e4e57c33f'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:25:12.837833 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-f53cb132-94a2-44e8-b9d5-659a33fee2e6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '329d65c2d27c4f9fb13d417e4e57c33f'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:25:12.844796 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-f53cb132-94a2-44e8-b9d5-659a33fee2e6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '329d65c2d27c4f9fb13d417e4e57c33f'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:25:12.851524 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1014/2005] 10.222.0.15 () {58 vars in 1224 bytes} [Thu Sep 15 12:25:12 2022] DELETE /identity/v3/users/329d65c2d27c4f9fb13d417e4e57c33f => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:25:12.856451 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b8fb9e-18b6-4e9d-afa7-3bfe3578ee9c None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:25:12.857033 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b8fb9e-18b6-4e9d-afa7-3bfe3578ee9c None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:25:12.857244 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b8fb9e-18b6-4e9d-afa7-3bfe3578ee9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:25:12.878347 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-10b8fb9e-18b6-4e9d-afa7-3bfe3578ee9c None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:25:12.880115 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-10b8fb9e-18b6-4e9d-afa7-3bfe3578ee9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:25:12.942477 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 992/2006] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 12:25:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:25:12.952669 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53cb132-94a2-44e8-b9d5-659a33fee2e6 admin admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:25:12.954337 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-f53cb132-94a2-44e8-b9d5-659a33fee2e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:25:13.016557 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53cb132-94a2-44e8-b9d5-659a33fee2e6 admin admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:25:13.020236 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e422cf9-ccbf-4df6-9f9e-fb8bd040fe90 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfbb4d6809064f9cb9e06626ed7d6814', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '540a32ef88764e8c8d9201ae22f6ddaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:25:13.020907 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e422cf9-ccbf-4df6-9f9e-fb8bd040fe90 service neutron] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:25:13.021188 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e422cf9-ccbf-4df6-9f9e-fb8bd040fe90 service neutron] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:25:13.021465 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e422cf9-ccbf-4df6-9f9e-fb8bd040fe90 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:25:13.030391 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e422cf9-ccbf-4df6-9f9e-fb8bd040fe90 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:25:13.031815 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e422cf9-ccbf-4df6-9f9e-fb8bd040fe90 service neutron] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:25:13.042045 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1015/2007] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 12:25:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 12:25:13.338754 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b8fb9e-18b6-4e9d-afa7-3bfe3578ee9c None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:25:13.340536 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-10b8fb9e-18b6-4e9d-afa7-3bfe3578ee9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:25:13.355139 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b8fb9e-18b6-4e9d-afa7-3bfe3578ee9c None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:25:13.360363 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d85949-eaf2-4081-8f07-cfe783c11af3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:25:13.361153 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d85949-eaf2-4081-8f07-cfe783c11af3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:25:13.369983 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d85949-eaf2-4081-8f07-cfe783c11af3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:25:13.370428 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0d85949-eaf2-4081-8f07-cfe783c11af3 admin admin] PATH_INFO: `/v3/projects/4c5c7aeb70f148ad83169665e6fa6f32` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:25:13.371827 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d85949-eaf2-4081-8f07-cfe783c11af3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4c5c7aeb70f148ad83169665e6fa6f32)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:25:13.374009 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0d85949-eaf2-4081-8f07-cfe783c11af3 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:25:13.424059 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-c0d85949-eaf2-4081-8f07-cfe783c11af3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4c5c7aeb70f148ad83169665e6fa6f32', 'request_id': 'req-c0d85949-eaf2-4081-8f07-cfe783c11af3'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:25:13.431221 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-c0d85949-eaf2-4081-8f07-cfe783c11af3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4c5c7aeb70f148ad83169665e6fa6f32', 'request_id': 'req-c0d85949-eaf2-4081-8f07-cfe783c11af3'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:25:13.436746 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-c0d85949-eaf2-4081-8f07-cfe783c11af3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4c5c7aeb70f148ad83169665e6fa6f32', 'request_id': 'req-c0d85949-eaf2-4081-8f07-cfe783c11af3'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:25:13.444363 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:25:13.449398 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-c0d85949-eaf2-4081-8f07-cfe783c11af3 admin admin] The token cache is being invalidate because project 4c5c7aeb70f148ad83169665e6fa6f32 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81448) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 12:25:13.449730 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-c0d85949-eaf2-4081-8f07-cfe783c11af3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:25:13.450473 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-c0d85949-eaf2-4081-8f07-cfe783c11af3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:25:13.452050 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 993/2008] 10.222.0.15 () {58 vars in 1233 bytes} [Thu Sep 15 12:25:13 2022] DELETE /identity/v3/projects/4c5c7aeb70f148ad83169665e6fa6f32 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:25:13.456319 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e422cf9-ccbf-4df6-9f9e-fb8bd040fe90 service neutron] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:25:13.466554 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e422cf9-ccbf-4df6-9f9e-fb8bd040fe90 service neutron] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:25:13.525095 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af7d3e7-7646-4bea-ba74-17a93ebaab97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:25:13.525736 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af7d3e7-7646-4bea-ba74-17a93ebaab97 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:25:13.526017 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af7d3e7-7646-4bea-ba74-17a93ebaab97 admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:25:13.526347 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af7d3e7-7646-4bea-ba74-17a93ebaab97 admin admin] PATH_INFO: `/v3/users/8ea8dce4c7b24a06b27b96f8089130aa` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:25:13.527529 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af7d3e7-7646-4bea-ba74-17a93ebaab97 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8ea8dce4c7b24a06b27b96f8089130aa)` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:25:13.529484 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af7d3e7-7646-4bea-ba74-17a93ebaab97 admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:25:13.563164 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:25:13.578986 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-8af7d3e7-7646-4bea-ba74-17a93ebaab97 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8ea8dce4c7b24a06b27b96f8089130aa'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:25:13.586088 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-8af7d3e7-7646-4bea-ba74-17a93ebaab97 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8ea8dce4c7b24a06b27b96f8089130aa'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:25:13.598853 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-8af7d3e7-7646-4bea-ba74-17a93ebaab97 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8ea8dce4c7b24a06b27b96f8089130aa'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:25:13.599518 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-8af7d3e7-7646-4bea-ba74-17a93ebaab97 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8ea8dce4c7b24a06b27b96f8089130aa'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:25:13.606998 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-8af7d3e7-7646-4bea-ba74-17a93ebaab97 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8ea8dce4c7b24a06b27b96f8089130aa'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:25:13.613081 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1016/2009] 10.222.0.15 () {58 vars in 1224 bytes} [Thu Sep 15 12:25:13 2022] DELETE /identity/v3/users/8ea8dce4c7b24a06b27b96f8089130aa => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:25:13.901526 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d85949-eaf2-4081-8f07-cfe783c11af3 admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:25:13.903229 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-c0d85949-eaf2-4081-8f07-cfe783c11af3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:25:13.923698 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0d85949-eaf2-4081-8f07-cfe783c11af3 admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:25:14.006863 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e823fc87-e95b-454a-b0bc-bf6193042bda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:25:14.007531 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e823fc87-e95b-454a-b0bc-bf6193042bda admin admin] REQUEST_METHOD: `DELETE` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:25:14.007807 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e823fc87-e95b-454a-b0bc-bf6193042bda admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:25:14.008076 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e823fc87-e95b-454a-b0bc-bf6193042bda admin admin] PATH_INFO: `/v3/projects/140d281723af41beb956c46aecbafe6a` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:25:14.009795 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e823fc87-e95b-454a-b0bc-bf6193042bda admin admin] RBAC: Authorizing `identity:delete_project(project_id=140d281723af41beb956c46aecbafe6a)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:25:14.011880 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e823fc87-e95b-454a-b0bc-bf6193042bda admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:25:14.058085 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-e823fc87-e95b-454a-b0bc-bf6193042bda admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '140d281723af41beb956c46aecbafe6a', 'request_id': 'req-e823fc87-e95b-454a-b0bc-bf6193042bda'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:25:14.066469 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-e823fc87-e95b-454a-b0bc-bf6193042bda admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '140d281723af41beb956c46aecbafe6a', 'request_id': 'req-e823fc87-e95b-454a-b0bc-bf6193042bda'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:25:14.072702 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-e823fc87-e95b-454a-b0bc-bf6193042bda admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '140d281723af41beb956c46aecbafe6a', 'request_id': 'req-e823fc87-e95b-454a-b0bc-bf6193042bda'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:25:14.083960 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-e823fc87-e95b-454a-b0bc-bf6193042bda admin admin] The token cache is being invalidate because project 140d281723af41beb956c46aecbafe6a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81448) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 12:25:14.084248 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-e823fc87-e95b-454a-b0bc-bf6193042bda admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:25:14.084939 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-e823fc87-e95b-454a-b0bc-bf6193042bda admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:25:14.086459 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 994/2010] 10.222.0.15 () {58 vars in 1233 bytes} [Thu Sep 15 12:25:13 2022] DELETE /identity/v3/projects/140d281723af41beb956c46aecbafe6a => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:25:14.091467 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aba3970-faa3-4cb9-a58b-bfb5392ede5d None None] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:25:14.091914 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aba3970-faa3-4cb9-a58b-bfb5392ede5d None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:25:14.092024 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aba3970-faa3-4cb9-a58b-bfb5392ede5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:25:14.111461 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.auth.core [None req-8aba3970-faa3-4cb9-a58b-bfb5392ede5d None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81449) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:25:14.112656 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-8aba3970-faa3-4cb9-a58b-bfb5392ede5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:25:14.171759 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1017/2011] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 12:25:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:25:14.179536 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379653d7-81fb-454d-931d-8a7f453914e1 None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:25:14.179821 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379653d7-81fb-454d-931d-8a7f453914e1 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:25:14.180092 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379653d7-81fb-454d-931d-8a7f453914e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:25:14.200493 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-379653d7-81fb-454d-931d-8a7f453914e1 None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:25:14.201703 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-379653d7-81fb-454d-931d-8a7f453914e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:25:14.209662 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 995/2012] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 12:25:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:25:14.213836 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aba3970-faa3-4cb9-a58b-bfb5392ede5d None None] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:25:14.223516 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aba3970-faa3-4cb9-a58b-bfb5392ede5d None None] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:25:14.228582 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680406a8-55a3-4ca8-926b-fd1615ca03ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:25:14.229217 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680406a8-55a3-4ca8-926b-fd1615ca03ae admin admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:25:14.229499 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680406a8-55a3-4ca8-926b-fd1615ca03ae admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:25:14.229770 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680406a8-55a3-4ca8-926b-fd1615ca03ae admin admin] PATH_INFO: `/v3/domains` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:25:14.230412 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680406a8-55a3-4ca8-926b-fd1615ca03ae admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81449) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:25:14.230713 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680406a8-55a3-4ca8-926b-fd1615ca03ae admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:25:14.232290 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680406a8-55a3-4ca8-926b-fd1615ca03ae admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:25:14.244021 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1018/2013] 10.222.0.15 () {58 vars in 1153 bytes} [Thu Sep 15 12:25:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 12:25:14.248102 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379653d7-81fb-454d-931d-8a7f453914e1 None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:25:14.257346 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379653d7-81fb-454d-931d-8a7f453914e1 None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:25:14.263464 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476c05c6-efb8-4536-b32a-6546670dbfe7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:25:14.264058 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476c05c6-efb8-4536-b32a-6546670dbfe7 admin admin] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:25:14.264333 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476c05c6-efb8-4536-b32a-6546670dbfe7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:25:14.264604 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476c05c6-efb8-4536-b32a-6546670dbfe7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:25:14.265288 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476c05c6-efb8-4536-b32a-6546670dbfe7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:25:14.267195 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476c05c6-efb8-4536-b32a-6546670dbfe7 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:25:14.294882 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 996/2014] 10.222.0.15 () {60 vars in 1153 bytes} [Thu Sep 15 12:25:14 2022] POST /identity/v3/projects => generated 385 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 12:25:14.298875 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680406a8-55a3-4ca8-926b-fd1615ca03ae admin admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:25:14.309483 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680406a8-55a3-4ca8-926b-fd1615ca03ae admin admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:25:14.369580 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd48ab7-8eb5-4fd6-9a43-93e82de77795 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:25:14.370222 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd48ab7-8eb5-4fd6-9a43-93e82de77795 admin admin] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:25:14.370499 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd48ab7-8eb5-4fd6-9a43-93e82de77795 admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:25:14.370768 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd48ab7-8eb5-4fd6-9a43-93e82de77795 admin admin] PATH_INFO: `/v3/users` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:25:14.371404 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd48ab7-8eb5-4fd6-9a43-93e82de77795 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:25:14.373344 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd48ab7-8eb5-4fd6-9a43-93e82de77795 admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:25:14.394052 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1019/2015] 10.222.0.15 () {60 vars in 1144 bytes} [Thu Sep 15 12:25:14 2022] POST /identity/v3/users => generated 293 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 15 12:25:14.398310 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476c05c6-efb8-4536-b32a-6546670dbfe7 admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:25:14.408700 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476c05c6-efb8-4536-b32a-6546670dbfe7 admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:25:14.413544 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370bd318-2c3e-4ca6-9501-1abe5bc52e0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:25:14.414192 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370bd318-2c3e-4ca6-9501-1abe5bc52e0e admin admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:25:14.414473 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370bd318-2c3e-4ca6-9501-1abe5bc52e0e admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:25:14.415246 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-370bd318-2c3e-4ca6-9501-1abe5bc52e0e admin admin] PATH_INFO: `/v3/roles` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:25:14.415866 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-370bd318-2c3e-4ca6-9501-1abe5bc52e0e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:25:14.418402 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-370bd318-2c3e-4ca6-9501-1abe5bc52e0e admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:25:14.431015 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 997/2016] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 12:25:14 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 12:25:14.435587 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd48ab7-8eb5-4fd6-9a43-93e82de77795 admin admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:25:14.446841 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd48ab7-8eb5-4fd6-9a43-93e82de77795 admin admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:25:14.451597 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b11db2a-2e64-4a9e-bfb1-45e3ce263c1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:25:14.452212 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b11db2a-2e64-4a9e-bfb1-45e3ce263c1f admin admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:25:14.452492 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b11db2a-2e64-4a9e-bfb1-45e3ce263c1f admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:25:14.452778 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b11db2a-2e64-4a9e-bfb1-45e3ce263c1f admin admin] PATH_INFO: `/v3/roles` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:25:14.453564 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b11db2a-2e64-4a9e-bfb1-45e3ce263c1f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:25:14.455158 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b11db2a-2e64-4a9e-bfb1-45e3ce263c1f admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:25:14.465944 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1020/2017] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 12:25:14 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 12:25:14.470441 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370bd318-2c3e-4ca6-9501-1abe5bc52e0e admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:25:14.482655 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-370bd318-2c3e-4ca6-9501-1abe5bc52e0e admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:25:14.487428 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5241a1-f109-4b61-8b2c-95106338c276 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:25:14.488088 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5241a1-f109-4b61-8b2c-95106338c276 admin admin] REQUEST_METHOD: `PUT` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:25:14.488390 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5241a1-f109-4b61-8b2c-95106338c276 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:25:14.488681 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5241a1-f109-4b61-8b2c-95106338c276 admin admin] PATH_INFO: `/v3/projects/f2d7b891b7a947bf81841d41556ace73/users/7afeafca9c2e4deb87edc12eb9d39698/roles/0679d6628a25438d90dc4bdc0401eda2` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:25:14.508874 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e5241a1-f109-4b61-8b2c-95106338c276 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f2d7b891b7a947bf81841d41556ace73, user_id=7afeafca9c2e4deb87edc12eb9d39698, role_id=0679d6628a25438d90dc4bdc0401eda2)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:25:14.511708 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e5241a1-f109-4b61-8b2c-95106338c276 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:25:14.525647 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 998/2018] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 15 12:25:14 2022] PUT /identity/v3/projects/f2d7b891b7a947bf81841d41556ace73/users/7afeafca9c2e4deb87edc12eb9d39698/roles/0679d6628a25438d90dc4bdc0401eda2 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:25:14.535974 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6a4998-b13d-4a8f-9f34-320d6594146f None None] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:25:14.536264 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6a4998-b13d-4a8f-9f34-320d6594146f None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:25:14.536564 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6a4998-b13d-4a8f-9f34-320d6594146f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:25:14.562652 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:25:14.565691 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.auth.core [None req-db6a4998-b13d-4a8f-9f34-320d6594146f None None] MFA Rules not processed for user `7afeafca9c2e4deb87edc12eb9d39698`. Rule list: `[]` (Enabled: `True`). {{(pid=81449) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:25:14.567428 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-db6a4998-b13d-4a8f-9f34-320d6594146f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:25:14.631942 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1021/2019] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 12:25:14 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:25:14.643768 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5241a1-f109-4b61-8b2c-95106338c276 admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:25:14.645447 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-8e5241a1-f109-4b61-8b2c-95106338c276 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:25:14.679052 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:25:14.706278 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5241a1-f109-4b61-8b2c-95106338c276 admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:25:14.709846 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48c1274-aeef-48eb-98a7-15a71b793fd4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8191d3ccb874529b32f8d0026db629d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '540a32ef88764e8c8d9201ae22f6ddaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:25:14.710449 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48c1274-aeef-48eb-98a7-15a71b793fd4 service cinder] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:25:14.710726 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48c1274-aeef-48eb-98a7-15a71b793fd4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:25:14.710995 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48c1274-aeef-48eb-98a7-15a71b793fd4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:25:14.719781 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f48c1274-aeef-48eb-98a7-15a71b793fd4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:25:14.721504 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f48c1274-aeef-48eb-98a7-15a71b793fd4 service cinder] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:25:14.730614 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 999/2020] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 12:25:14 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 12:25:14.749595 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6a4998-b13d-4a8f-9f34-320d6594146f None None] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:25:14.816263 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6a4998-b13d-4a8f-9f34-320d6594146f None None] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:25:14.820713 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddcc9c4a-f85a-42a1-bbdf-be6446f45d85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:25:14.821400 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcc9c4a-f85a-42a1-bbdf-be6446f45d85 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:25:14.821675 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcc9c4a-f85a-42a1-bbdf-be6446f45d85 admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:25:14.821943 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcc9c4a-f85a-42a1-bbdf-be6446f45d85 admin admin] PATH_INFO: `/v3/users/7afeafca9c2e4deb87edc12eb9d39698` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:25:14.823039 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddcc9c4a-f85a-42a1-bbdf-be6446f45d85 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7afeafca9c2e4deb87edc12eb9d39698)` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:25:14.824968 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddcc9c4a-f85a-42a1-bbdf-be6446f45d85 admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:25:14.869900 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-ddcc9c4a-f85a-42a1-bbdf-be6446f45d85 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7afeafca9c2e4deb87edc12eb9d39698'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:25:14.878014 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-ddcc9c4a-f85a-42a1-bbdf-be6446f45d85 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7afeafca9c2e4deb87edc12eb9d39698'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:25:14.889723 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-ddcc9c4a-f85a-42a1-bbdf-be6446f45d85 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7afeafca9c2e4deb87edc12eb9d39698'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:25:14.890368 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-ddcc9c4a-f85a-42a1-bbdf-be6446f45d85 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7afeafca9c2e4deb87edc12eb9d39698'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:25:14.897104 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-ddcc9c4a-f85a-42a1-bbdf-be6446f45d85 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7afeafca9c2e4deb87edc12eb9d39698'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:25:14.903384 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1022/2021] 10.222.0.15 () {58 vars in 1224 bytes} [Thu Sep 15 12:25:14 2022] DELETE /identity/v3/users/7afeafca9c2e4deb87edc12eb9d39698 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:25:14.907876 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed6a838-91d4-4e0d-b946-47c9ad60a494 None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:25:14.908325 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed6a838-91d4-4e0d-b946-47c9ad60a494 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:25:14.908444 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed6a838-91d4-4e0d-b946-47c9ad60a494 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:25:14.927629 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-bed6a838-91d4-4e0d-b946-47c9ad60a494 None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:25:14.928840 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-bed6a838-91d4-4e0d-b946-47c9ad60a494 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:25:14.987625 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1000/2022] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 12:25:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:25:14.997751 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddcc9c4a-f85a-42a1-bbdf-be6446f45d85 admin admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:25:14.999216 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-ddcc9c4a-f85a-42a1-bbdf-be6446f45d85 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:25:15.028157 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:25:15.066311 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddcc9c4a-f85a-42a1-bbdf-be6446f45d85 admin admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:25:15.069935 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c8c8d9-2266-4d38-b6b7-af3843647c6f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfbb4d6809064f9cb9e06626ed7d6814', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '540a32ef88764e8c8d9201ae22f6ddaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:25:15.070602 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c8c8d9-2266-4d38-b6b7-af3843647c6f service neutron] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:25:15.070893 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c8c8d9-2266-4d38-b6b7-af3843647c6f service neutron] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:25:15.071163 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c8c8d9-2266-4d38-b6b7-af3843647c6f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:25:15.080570 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c8c8d9-2266-4d38-b6b7-af3843647c6f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:25:15.082052 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57c8c8d9-2266-4d38-b6b7-af3843647c6f service neutron] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:25:15.092847 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1023/2023] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 12:25:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 12:25:15.358802 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed6a838-91d4-4e0d-b946-47c9ad60a494 None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:25:15.360434 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-bed6a838-91d4-4e0d-b946-47c9ad60a494 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:25:15.380990 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed6a838-91d4-4e0d-b946-47c9ad60a494 None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:25:15.393107 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59bc6b12-15a3-403b-831b-677c34ee6041 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:25:15.393746 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bc6b12-15a3-403b-831b-677c34ee6041 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:25:15.394023 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bc6b12-15a3-403b-831b-677c34ee6041 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:25:15.394290 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bc6b12-15a3-403b-831b-677c34ee6041 admin admin] PATH_INFO: `/v3/projects/f2d7b891b7a947bf81841d41556ace73` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:25:15.395406 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59bc6b12-15a3-403b-831b-677c34ee6041 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f2d7b891b7a947bf81841d41556ace73)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:25:15.397388 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59bc6b12-15a3-403b-831b-677c34ee6041 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:25:15.445790 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-59bc6b12-15a3-403b-831b-677c34ee6041 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f2d7b891b7a947bf81841d41556ace73', 'request_id': 'req-59bc6b12-15a3-403b-831b-677c34ee6041'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:25:15.452785 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-59bc6b12-15a3-403b-831b-677c34ee6041 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f2d7b891b7a947bf81841d41556ace73', 'request_id': 'req-59bc6b12-15a3-403b-831b-677c34ee6041'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:25:15.458316 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-59bc6b12-15a3-403b-831b-677c34ee6041 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f2d7b891b7a947bf81841d41556ace73', 'request_id': 'req-59bc6b12-15a3-403b-831b-677c34ee6041'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:25:15.469454 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-59bc6b12-15a3-403b-831b-677c34ee6041 admin admin] The token cache is being invalidate because project f2d7b891b7a947bf81841d41556ace73 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81448) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 12:25:15.469743 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-59bc6b12-15a3-403b-831b-677c34ee6041 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:25:15.470616 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-59bc6b12-15a3-403b-831b-677c34ee6041 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:25:15.472100 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1001/2024] 10.222.0.15 () {58 vars in 1233 bytes} [Thu Sep 15 12:25:15 2022] DELETE /identity/v3/projects/f2d7b891b7a947bf81841d41556ace73 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:25:15.477106 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33734c13-26b0-48d1-85d8-f00dcfed4ad2 None None] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:25:15.477392 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33734c13-26b0-48d1-85d8-f00dcfed4ad2 None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:25:15.477665 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33734c13-26b0-48d1-85d8-f00dcfed4ad2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:25:15.497101 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.auth.core [None req-33734c13-26b0-48d1-85d8-f00dcfed4ad2 None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81449) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:25:15.498214 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-33734c13-26b0-48d1-85d8-f00dcfed4ad2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:25:15.554757 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1024/2025] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 12:25:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:25:15.563548 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98b888d-3288-4b2c-b11b-fd207387abd1 None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:25:15.563736 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98b888d-3288-4b2c-b11b-fd207387abd1 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:25:15.564000 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98b888d-3288-4b2c-b11b-fd207387abd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:25:15.583312 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-b98b888d-3288-4b2c-b11b-fd207387abd1 None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:25:15.584494 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-b98b888d-3288-4b2c-b11b-fd207387abd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:25:15.592831 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1002/2026] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 12:25:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:25:15.597447 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33734c13-26b0-48d1-85d8-f00dcfed4ad2 None None] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:25:15.607616 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33734c13-26b0-48d1-85d8-f00dcfed4ad2 None None] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:25:15.612212 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4810d504-4d89-4bb8-839e-aa7e4832c542 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:25:15.612834 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4810d504-4d89-4bb8-839e-aa7e4832c542 admin admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:25:15.613118 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4810d504-4d89-4bb8-839e-aa7e4832c542 admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:25:15.613389 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4810d504-4d89-4bb8-839e-aa7e4832c542 admin admin] PATH_INFO: `/v3/domains` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:25:15.614028 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4810d504-4d89-4bb8-839e-aa7e4832c542 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81449) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:25:15.614331 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4810d504-4d89-4bb8-839e-aa7e4832c542 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:25:15.615902 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4810d504-4d89-4bb8-839e-aa7e4832c542 admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:25:15.627178 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1025/2027] 10.222.0.15 () {58 vars in 1153 bytes} [Thu Sep 15 12:25:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:25:15.631305 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98b888d-3288-4b2c-b11b-fd207387abd1 None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:25:15.640314 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b98b888d-3288-4b2c-b11b-fd207387abd1 None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:25:15.645072 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19485a1d-8419-4cd9-9b31-65133d872ea5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:25:15.645673 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19485a1d-8419-4cd9-9b31-65133d872ea5 admin admin] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:25:15.645949 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19485a1d-8419-4cd9-9b31-65133d872ea5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:25:15.646215 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19485a1d-8419-4cd9-9b31-65133d872ea5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:25:15.646837 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19485a1d-8419-4cd9-9b31-65133d872ea5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:25:15.648867 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19485a1d-8419-4cd9-9b31-65133d872ea5 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:25:15.678067 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1003/2028] 10.222.0.15 () {60 vars in 1153 bytes} [Thu Sep 15 12:25:15 2022] POST /identity/v3/projects => generated 383 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 12:25:15.682088 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4810d504-4d89-4bb8-839e-aa7e4832c542 admin admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:25:15.692026 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4810d504-4d89-4bb8-839e-aa7e4832c542 admin admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:25:15.746717 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7e0c71-a1c4-4f8c-be48-32f7ee572bd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:25:15.747369 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7e0c71-a1c4-4f8c-be48-32f7ee572bd0 admin admin] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:25:15.747592 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7e0c71-a1c4-4f8c-be48-32f7ee572bd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:25:15.747860 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7e0c71-a1c4-4f8c-be48-32f7ee572bd0 admin admin] PATH_INFO: `/v3/users` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:25:15.748473 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d7e0c71-a1c4-4f8c-be48-32f7ee572bd0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:25:15.750402 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d7e0c71-a1c4-4f8c-be48-32f7ee572bd0 admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:25:15.771335 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1026/2029] 10.222.0.15 () {60 vars in 1144 bytes} [Thu Sep 15 12:25:15 2022] POST /identity/v3/users => generated 292 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 12:25:15.776192 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19485a1d-8419-4cd9-9b31-65133d872ea5 admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:25:15.786137 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19485a1d-8419-4cd9-9b31-65133d872ea5 admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:25:15.790653 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5d8ac9-1cd3-4e61-8171-3ca67d7a1228 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:25:15.791247 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5d8ac9-1cd3-4e61-8171-3ca67d7a1228 admin admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:25:15.791522 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5d8ac9-1cd3-4e61-8171-3ca67d7a1228 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:25:15.791790 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5d8ac9-1cd3-4e61-8171-3ca67d7a1228 admin admin] PATH_INFO: `/v3/roles` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:25:15.792416 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e5d8ac9-1cd3-4e61-8171-3ca67d7a1228 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:25:15.794038 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e5d8ac9-1cd3-4e61-8171-3ca67d7a1228 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:25:15.803558 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1004/2030] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 12:25:15 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 12:25:15.807286 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7e0c71-a1c4-4f8c-be48-32f7ee572bd0 admin admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:25:15.817440 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7e0c71-a1c4-4f8c-be48-32f7ee572bd0 admin admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:25:15.822693 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e72ea5f-63bc-44d5-881f-84142474ec6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:25:15.823278 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e72ea5f-63bc-44d5-881f-84142474ec6b admin admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:25:15.823552 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e72ea5f-63bc-44d5-881f-84142474ec6b admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:25:15.823819 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e72ea5f-63bc-44d5-881f-84142474ec6b admin admin] PATH_INFO: `/v3/roles` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:25:15.824451 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e72ea5f-63bc-44d5-881f-84142474ec6b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:25:15.826064 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e72ea5f-63bc-44d5-881f-84142474ec6b admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:25:15.836295 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1027/2031] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 12:25:15 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 15 12:25:15.841019 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5d8ac9-1cd3-4e61-8171-3ca67d7a1228 admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:25:15.850925 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5d8ac9-1cd3-4e61-8171-3ca67d7a1228 admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:25:15.855438 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c12f945-6214-447b-8dd7-79ed6014f513 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:25:15.856087 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c12f945-6214-447b-8dd7-79ed6014f513 admin admin] REQUEST_METHOD: `PUT` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:25:15.856363 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c12f945-6214-447b-8dd7-79ed6014f513 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:25:15.856635 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c12f945-6214-447b-8dd7-79ed6014f513 admin admin] PATH_INFO: `/v3/projects/762613f23f714b489992a2dd539058d4/users/be01f210b2d643cb9678f4cf1802c23e/roles/0679d6628a25438d90dc4bdc0401eda2` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:25:15.867712 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:25:15.872343 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c12f945-6214-447b-8dd7-79ed6014f513 admin admin] RBAC: Authorizing `identity:create_grant(project_id=762613f23f714b489992a2dd539058d4, user_id=be01f210b2d643cb9678f4cf1802c23e, role_id=0679d6628a25438d90dc4bdc0401eda2)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:25:15.875463 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c12f945-6214-447b-8dd7-79ed6014f513 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:25:15.889279 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1005/2032] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 15 12:25:15 2022] PUT /identity/v3/projects/762613f23f714b489992a2dd539058d4/users/be01f210b2d643cb9678f4cf1802c23e/roles/0679d6628a25438d90dc4bdc0401eda2 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:25:15.899265 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9da6241-f6b6-433b-86d5-38401c0b7c0e None None] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:25:15.899552 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9da6241-f6b6-433b-86d5-38401c0b7c0e None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:25:15.899826 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9da6241-f6b6-433b-86d5-38401c0b7c0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:25:15.927768 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.auth.core [None req-a9da6241-f6b6-433b-86d5-38401c0b7c0e None None] MFA Rules not processed for user `be01f210b2d643cb9678f4cf1802c23e`. Rule list: `[]` (Enabled: `True`). {{(pid=81449) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:25:15.928985 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-a9da6241-f6b6-433b-86d5-38401c0b7c0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:25:15.946341 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:25:15.994737 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1028/2033] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 12:25:15 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:25:15.999277 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c12f945-6214-447b-8dd7-79ed6014f513 admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:25:16.062136 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c12f945-6214-447b-8dd7-79ed6014f513 admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:25:16.066752 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2c644e-1840-4145-927b-65950c0d0e33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:25:16.067516 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2c644e-1840-4145-927b-65950c0d0e33 admin admin] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:25:16.067802 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2c644e-1840-4145-927b-65950c0d0e33 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:25:16.068072 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2c644e-1840-4145-927b-65950c0d0e33 admin admin] PATH_INFO: `/v3/projects` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:25:16.068698 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2c644e-1840-4145-927b-65950c0d0e33 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:25:16.070734 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2c644e-1840-4145-927b-65950c0d0e33 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:25:16.100403 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1006/2034] 10.222.0.15 () {60 vars in 1153 bytes} [Thu Sep 15 12:25:15 2022] POST /identity/v3/projects => generated 385 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 12:25:16.104660 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9da6241-f6b6-433b-86d5-38401c0b7c0e None None] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:25:16.115626 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9da6241-f6b6-433b-86d5-38401c0b7c0e None None] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:25:16.172982 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b405339d-7cf0-4e53-8d32-b6907a889ead admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:25:16.173597 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b405339d-7cf0-4e53-8d32-b6907a889ead admin admin] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:25:16.173872 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b405339d-7cf0-4e53-8d32-b6907a889ead admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:25:16.174139 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b405339d-7cf0-4e53-8d32-b6907a889ead admin admin] PATH_INFO: `/v3/users` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:25:16.174771 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b405339d-7cf0-4e53-8d32-b6907a889ead admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:25:16.176673 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b405339d-7cf0-4e53-8d32-b6907a889ead admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:25:16.198590 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1029/2035] 10.222.0.15 () {60 vars in 1144 bytes} [Thu Sep 15 12:25:16 2022] POST /identity/v3/users => generated 293 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 12:25:16.202710 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2c644e-1840-4145-927b-65950c0d0e33 admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:25:16.213233 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2c644e-1840-4145-927b-65950c0d0e33 admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:25:16.217927 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0c2b0e-6977-47e0-aaef-86e45d576230 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:25:16.218527 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0c2b0e-6977-47e0-aaef-86e45d576230 admin admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:25:16.218808 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0c2b0e-6977-47e0-aaef-86e45d576230 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:25:16.219081 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a0c2b0e-6977-47e0-aaef-86e45d576230 admin admin] PATH_INFO: `/v3/roles` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:25:16.219724 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0c2b0e-6977-47e0-aaef-86e45d576230 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:25:16.221610 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a0c2b0e-6977-47e0-aaef-86e45d576230 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:25:16.232053 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1007/2036] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 12:25:16 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 12:25:16.235906 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b405339d-7cf0-4e53-8d32-b6907a889ead admin admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:25:16.246174 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b405339d-7cf0-4e53-8d32-b6907a889ead admin admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:25:16.250716 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d02756-a3ca-44cf-b223-58e786441493 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:25:16.251245 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d02756-a3ca-44cf-b223-58e786441493 admin admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:25:16.251519 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d02756-a3ca-44cf-b223-58e786441493 admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:25:16.251782 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d02756-a3ca-44cf-b223-58e786441493 admin admin] PATH_INFO: `/v3/roles` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:25:16.252410 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d02756-a3ca-44cf-b223-58e786441493 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:25:16.254009 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d02756-a3ca-44cf-b223-58e786441493 admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:25:16.264354 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1030/2037] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 12:25:16 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 12:25:16.268285 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0c2b0e-6977-47e0-aaef-86e45d576230 admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:25:16.278239 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a0c2b0e-6977-47e0-aaef-86e45d576230 admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:25:16.283742 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef0488d-0cac-4b9a-9146-86b78801aa5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:25:16.283881 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef0488d-0cac-4b9a-9146-86b78801aa5c admin admin] REQUEST_METHOD: `PUT` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:25:16.284138 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef0488d-0cac-4b9a-9146-86b78801aa5c admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:25:16.284413 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef0488d-0cac-4b9a-9146-86b78801aa5c admin admin] PATH_INFO: `/v3/projects/dcd1fdb7a9824f59a5118e20466066d2/users/034c9609640740659ca1e86fd13f0044/roles/0679d6628a25438d90dc4bdc0401eda2` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:25:16.299856 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef0488d-0cac-4b9a-9146-86b78801aa5c admin admin] RBAC: Authorizing `identity:create_grant(project_id=dcd1fdb7a9824f59a5118e20466066d2, user_id=034c9609640740659ca1e86fd13f0044, role_id=0679d6628a25438d90dc4bdc0401eda2)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:25:16.302544 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef0488d-0cac-4b9a-9146-86b78801aa5c admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:25:16.315857 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1008/2038] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 15 12:25:16 2022] PUT /identity/v3/projects/dcd1fdb7a9824f59a5118e20466066d2/users/034c9609640740659ca1e86fd13f0044/roles/0679d6628a25438d90dc4bdc0401eda2 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:25:16.330417 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d175234-bfae-4e1b-80a8-cdca41877bdf None None] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:25:16.330706 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d175234-bfae-4e1b-80a8-cdca41877bdf None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:25:16.330983 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d175234-bfae-4e1b-80a8-cdca41877bdf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:25:16.359747 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.auth.core [None req-4d175234-bfae-4e1b-80a8-cdca41877bdf None None] MFA Rules not processed for user `034c9609640740659ca1e86fd13f0044`. Rule list: `[]` (Enabled: `True`). {{(pid=81449) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:25:16.361027 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-4d175234-bfae-4e1b-80a8-cdca41877bdf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:25:16.429004 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1031/2039] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 12:25:16 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:25:16.434099 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef0488d-0cac-4b9a-9146-86b78801aa5c admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:25:16.496044 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef0488d-0cac-4b9a-9146-86b78801aa5c admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:25:16.500801 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f8dfe0-f813-4c96-a23e-0aa4a0cc98af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:25:16.502473 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f8dfe0-f813-4c96-a23e-0aa4a0cc98af admin admin] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:25:16.502757 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f8dfe0-f813-4c96-a23e-0aa4a0cc98af admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:25:16.503031 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f8dfe0-f813-4c96-a23e-0aa4a0cc98af admin admin] PATH_INFO: `/v3/projects` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:25:16.503664 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f8dfe0-f813-4c96-a23e-0aa4a0cc98af admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:25:16.505593 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f8dfe0-f813-4c96-a23e-0aa4a0cc98af admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:25:16.532314 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1009/2040] 10.222.0.15 () {60 vars in 1153 bytes} [Thu Sep 15 12:25:16 2022] POST /identity/v3/projects => generated 383 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 12:25:16.536434 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d175234-bfae-4e1b-80a8-cdca41877bdf None None] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:25:16.547482 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d175234-bfae-4e1b-80a8-cdca41877bdf None None] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:25:16.575037 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:25:16.606942 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d616df-a320-4e19-9311-b1d83521a827 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:25:16.607546 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d616df-a320-4e19-9311-b1d83521a827 admin admin] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:25:16.607821 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d616df-a320-4e19-9311-b1d83521a827 admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:25:16.608086 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d616df-a320-4e19-9311-b1d83521a827 admin admin] PATH_INFO: `/v3/users` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:25:16.608722 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d616df-a320-4e19-9311-b1d83521a827 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:25:16.610756 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d616df-a320-4e19-9311-b1d83521a827 admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:25:16.635021 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1032/2041] 10.222.0.15 () {60 vars in 1144 bytes} [Thu Sep 15 12:25:16 2022] POST /identity/v3/users => generated 298 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 12:25:16.639130 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f8dfe0-f813-4c96-a23e-0aa4a0cc98af admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:25:16.651104 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f8dfe0-f813-4c96-a23e-0aa4a0cc98af admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:25:16.658614 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37b9f90-22b2-45fe-b161-cebadd066157 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:25:16.659317 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37b9f90-22b2-45fe-b161-cebadd066157 admin admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:25:16.659599 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37b9f90-22b2-45fe-b161-cebadd066157 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:25:16.660055 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37b9f90-22b2-45fe-b161-cebadd066157 admin admin] PATH_INFO: `/v3/roles` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:25:16.661243 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37b9f90-22b2-45fe-b161-cebadd066157 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:25:16.664872 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37b9f90-22b2-45fe-b161-cebadd066157 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:25:16.679856 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1010/2042] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 12:25:16 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 12:25:16.681067 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d616df-a320-4e19-9311-b1d83521a827 admin admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:25:16.691977 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d616df-a320-4e19-9311-b1d83521a827 admin admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:25:16.696604 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e7827a-db41-45d9-9afe-7cc28de114bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:25:16.697326 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e7827a-db41-45d9-9afe-7cc28de114bc admin admin] REQUEST_METHOD: `PUT` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:25:16.697601 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e7827a-db41-45d9-9afe-7cc28de114bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:25:16.697871 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e7827a-db41-45d9-9afe-7cc28de114bc admin admin] PATH_INFO: `/v3/projects/fad3b3ceac6f454d9e05dec496e58eec/users/01f36333b8fa4550b42e588eabffb783/roles/9c769b5c82884766b2c510b1f0f1364c` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:25:16.712682 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e7827a-db41-45d9-9afe-7cc28de114bc admin admin] RBAC: Authorizing `identity:create_grant(project_id=fad3b3ceac6f454d9e05dec496e58eec, user_id=01f36333b8fa4550b42e588eabffb783, role_id=9c769b5c82884766b2c510b1f0f1364c)` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:25:16.715340 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e7827a-db41-45d9-9afe-7cc28de114bc admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:25:16.728780 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1033/2043] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 15 12:25:16 2022] PUT /identity/v3/projects/fad3b3ceac6f454d9e05dec496e58eec/users/01f36333b8fa4550b42e588eabffb783/roles/9c769b5c82884766b2c510b1f0f1364c => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 12:25:16.732815 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37b9f90-22b2-45fe-b161-cebadd066157 admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:25:16.742936 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37b9f90-22b2-45fe-b161-cebadd066157 admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:25:16.753461 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:25:16.807765 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9bbf0e-365f-490b-a7bc-2b4401e53ad8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:25:16.808469 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9bbf0e-365f-490b-a7bc-2b4401e53ad8 admin admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:25:16.808779 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9bbf0e-365f-490b-a7bc-2b4401e53ad8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:25:16.809059 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9bbf0e-365f-490b-a7bc-2b4401e53ad8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:25:16.809696 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9bbf0e-365f-490b-a7bc-2b4401e53ad8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:25:16.812426 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9bbf0e-365f-490b-a7bc-2b4401e53ad8 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:25:16.822763 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1011/2044] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 12:25:16 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 12:25:16.826901 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e7827a-db41-45d9-9afe-7cc28de114bc admin admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:25:16.837228 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e7827a-db41-45d9-9afe-7cc28de114bc admin admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:25:16.842347 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c46c817-2acc-4de2-8612-877ecfe58286 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:25:16.842991 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c46c817-2acc-4de2-8612-877ecfe58286 admin admin] REQUEST_METHOD: `PUT` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:25:16.843271 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c46c817-2acc-4de2-8612-877ecfe58286 admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:25:16.843542 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c46c817-2acc-4de2-8612-877ecfe58286 admin admin] PATH_INFO: `/v3/projects/fad3b3ceac6f454d9e05dec496e58eec/users/01f36333b8fa4550b42e588eabffb783/roles/9c769b5c82884766b2c510b1f0f1364c` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:25:16.845725 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c46c817-2acc-4de2-8612-877ecfe58286 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fad3b3ceac6f454d9e05dec496e58eec, user_id=01f36333b8fa4550b42e588eabffb783, role_id=9c769b5c82884766b2c510b1f0f1364c)` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:25:16.848335 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c46c817-2acc-4de2-8612-877ecfe58286 admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:25:16.859132 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1034/2045] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 15 12:25:16 2022] PUT /identity/v3/projects/fad3b3ceac6f454d9e05dec496e58eec/users/01f36333b8fa4550b42e588eabffb783/roles/9c769b5c82884766b2c510b1f0f1364c => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:25:16.926819 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66e8f5a-5dc1-402d-a2f6-d0ced017f916 None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:25:16.927281 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66e8f5a-5dc1-402d-a2f6-d0ced017f916 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:25:16.927724 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66e8f5a-5dc1-402d-a2f6-d0ced017f916 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:25:16.955571 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-a66e8f5a-5dc1-402d-a2f6-d0ced017f916 None None] MFA Rules not processed for user `01f36333b8fa4550b42e588eabffb783`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:25:16.956802 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-a66e8f5a-5dc1-402d-a2f6-d0ced017f916 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:25:17.034858 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1012/2046] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 12:25:16 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:25:17.046871 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c46c817-2acc-4de2-8612-877ecfe58286 admin admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:25:17.048706 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-2c46c817-2acc-4de2-8612-877ecfe58286 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:25:17.109890 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c46c817-2acc-4de2-8612-877ecfe58286 admin admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:25:17.113723 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3058e3-20e0-46e7-93d0-bfb9b011922a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8191d3ccb874529b32f8d0026db629d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '540a32ef88764e8c8d9201ae22f6ddaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:25:17.114333 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3058e3-20e0-46e7-93d0-bfb9b011922a service cinder] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:25:17.114612 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3058e3-20e0-46e7-93d0-bfb9b011922a service cinder] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:25:17.114884 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3058e3-20e0-46e7-93d0-bfb9b011922a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:25:17.172373 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3058e3-20e0-46e7-93d0-bfb9b011922a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:25:17.173851 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3058e3-20e0-46e7-93d0-bfb9b011922a service cinder] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:25:17.182724 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1035/2047] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 12:25:17 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 15 12:25:17.702939 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66e8f5a-5dc1-402d-a2f6-d0ced017f916 None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:25:17.704340 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-a66e8f5a-5dc1-402d-a2f6-d0ced017f916 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:25:17.768554 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66e8f5a-5dc1-402d-a2f6-d0ced017f916 None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:25:17.773056 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da29b8a4-a1c6-472e-8008-af53f79ce0e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:25:17.773689 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da29b8a4-a1c6-472e-8008-af53f79ce0e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:25:17.773965 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da29b8a4-a1c6-472e-8008-af53f79ce0e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:25:17.774234 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da29b8a4-a1c6-472e-8008-af53f79ce0e9 admin admin] PATH_INFO: `/v3/users/d63ca125537749399da8a22568641e13` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:25:17.775341 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da29b8a4-a1c6-472e-8008-af53f79ce0e9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d63ca125537749399da8a22568641e13)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:25:17.777269 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da29b8a4-a1c6-472e-8008-af53f79ce0e9 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:25:17.816025 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:25:17.826868 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-da29b8a4-a1c6-472e-8008-af53f79ce0e9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd63ca125537749399da8a22568641e13'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:25:17.841362 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-da29b8a4-a1c6-472e-8008-af53f79ce0e9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd63ca125537749399da8a22568641e13'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:25:17.850084 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-da29b8a4-a1c6-472e-8008-af53f79ce0e9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd63ca125537749399da8a22568641e13'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:25:17.850740 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-da29b8a4-a1c6-472e-8008-af53f79ce0e9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd63ca125537749399da8a22568641e13'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:25:17.858201 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-da29b8a4-a1c6-472e-8008-af53f79ce0e9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd63ca125537749399da8a22568641e13'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:25:17.863972 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1013/2048] 10.222.0.15 () {58 vars in 1224 bytes} [Thu Sep 15 12:25:17 2022] DELETE /identity/v3/users/d63ca125537749399da8a22568641e13 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:25:17.868396 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aad1592-e537-4146-bdea-eef3f7d9c373 None None] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:25:17.868687 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aad1592-e537-4146-bdea-eef3f7d9c373 None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:25:17.868983 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aad1592-e537-4146-bdea-eef3f7d9c373 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:25:17.888815 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.auth.core [None req-7aad1592-e537-4146-bdea-eef3f7d9c373 None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81449) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:25:17.890004 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-7aad1592-e537-4146-bdea-eef3f7d9c373 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:25:17.918680 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:25:17.947910 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1036/2049] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 12:25:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:25:17.957689 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da29b8a4-a1c6-472e-8008-af53f79ce0e9 admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:25:17.959090 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-da29b8a4-a1c6-472e-8008-af53f79ce0e9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:25:18.017454 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da29b8a4-a1c6-472e-8008-af53f79ce0e9 admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:25:18.021048 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8da147e-1147-4d79-af39-a095717f90c8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfbb4d6809064f9cb9e06626ed7d6814', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '540a32ef88764e8c8d9201ae22f6ddaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:25:18.021648 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8da147e-1147-4d79-af39-a095717f90c8 service neutron] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:25:18.021926 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8da147e-1147-4d79-af39-a095717f90c8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:25:18.022189 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8da147e-1147-4d79-af39-a095717f90c8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:25:18.030763 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8da147e-1147-4d79-af39-a095717f90c8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:25:18.032171 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8da147e-1147-4d79-af39-a095717f90c8 service neutron] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:25:18.043066 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1014/2050] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 12:25:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 12:25:18.339290 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aad1592-e537-4146-bdea-eef3f7d9c373 None None] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:25:18.340733 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-7aad1592-e537-4146-bdea-eef3f7d9c373 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:25:18.361547 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aad1592-e537-4146-bdea-eef3f7d9c373 None None] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:25:18.366952 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea7c6b3-f74c-4024-8933-de54354dd262 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:25:18.367601 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea7c6b3-f74c-4024-8933-de54354dd262 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:25:18.367884 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea7c6b3-f74c-4024-8933-de54354dd262 admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:25:18.368154 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea7c6b3-f74c-4024-8933-de54354dd262 admin admin] PATH_INFO: `/v3/projects/b230bd05a52240d8a4c8557d2d3bc5e1` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:25:18.369319 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea7c6b3-f74c-4024-8933-de54354dd262 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b230bd05a52240d8a4c8557d2d3bc5e1)` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:25:18.371328 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea7c6b3-f74c-4024-8933-de54354dd262 admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:25:18.430657 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-0ea7c6b3-f74c-4024-8933-de54354dd262 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b230bd05a52240d8a4c8557d2d3bc5e1', 'request_id': 'req-0ea7c6b3-f74c-4024-8933-de54354dd262'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:25:18.442609 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-0ea7c6b3-f74c-4024-8933-de54354dd262 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b230bd05a52240d8a4c8557d2d3bc5e1', 'request_id': 'req-0ea7c6b3-f74c-4024-8933-de54354dd262'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:25:18.452070 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-0ea7c6b3-f74c-4024-8933-de54354dd262 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b230bd05a52240d8a4c8557d2d3bc5e1', 'request_id': 'req-0ea7c6b3-f74c-4024-8933-de54354dd262'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:25:18.473922 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-0ea7c6b3-f74c-4024-8933-de54354dd262 admin admin] The token cache is being invalidate because project b230bd05a52240d8a4c8557d2d3bc5e1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81449) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 12:25:18.474214 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-0ea7c6b3-f74c-4024-8933-de54354dd262 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:25:18.474929 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-0ea7c6b3-f74c-4024-8933-de54354dd262 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:25:18.476777 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1037/2051] 10.222.0.15 () {58 vars in 1233 bytes} [Thu Sep 15 12:25:18 2022] DELETE /identity/v3/projects/b230bd05a52240d8a4c8557d2d3bc5e1 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:25:18.480533 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8da147e-1147-4d79-af39-a095717f90c8 service neutron] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:25:18.499262 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8da147e-1147-4d79-af39-a095717f90c8 service neutron] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:25:18.634018 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc0f1ba-904f-48ac-9641-0faa5b1a22d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:25:18.636272 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc0f1ba-904f-48ac-9641-0faa5b1a22d2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:25:18.637158 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc0f1ba-904f-48ac-9641-0faa5b1a22d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:25:18.637438 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc0f1ba-904f-48ac-9641-0faa5b1a22d2 admin admin] PATH_INFO: `/v3/users/2b59c1b53b7a47c5b2c2f084ef11f130` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:25:18.639142 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc0f1ba-904f-48ac-9641-0faa5b1a22d2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2b59c1b53b7a47c5b2c2f084ef11f130)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:25:18.642058 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc0f1ba-904f-48ac-9641-0faa5b1a22d2 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:25:18.727508 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-7bc0f1ba-904f-48ac-9641-0faa5b1a22d2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2b59c1b53b7a47c5b2c2f084ef11f130'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:25:18.737224 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:25:18.745638 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-7bc0f1ba-904f-48ac-9641-0faa5b1a22d2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2b59c1b53b7a47c5b2c2f084ef11f130'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:25:18.754085 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-7bc0f1ba-904f-48ac-9641-0faa5b1a22d2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2b59c1b53b7a47c5b2c2f084ef11f130'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:25:18.754903 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-7bc0f1ba-904f-48ac-9641-0faa5b1a22d2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2b59c1b53b7a47c5b2c2f084ef11f130'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:25:18.771376 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-7bc0f1ba-904f-48ac-9641-0faa5b1a22d2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2b59c1b53b7a47c5b2c2f084ef11f130'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:25:18.783401 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1015/2052] 10.222.0.15 () {58 vars in 1224 bytes} [Thu Sep 15 12:25:18 2022] DELETE /identity/v3/users/2b59c1b53b7a47c5b2c2f084ef11f130 => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:25:18.792745 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea7c6b3-f74c-4024-8933-de54354dd262 admin admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:25:18.794478 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-0ea7c6b3-f74c-4024-8933-de54354dd262 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:25:18.863040 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea7c6b3-f74c-4024-8933-de54354dd262 admin admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:25:18.921994 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c225b661-23fd-4af8-8cce-7afd9f03896d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8191d3ccb874529b32f8d0026db629d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '540a32ef88764e8c8d9201ae22f6ddaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:25:18.922638 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c225b661-23fd-4af8-8cce-7afd9f03896d service cinder] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:25:18.922912 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c225b661-23fd-4af8-8cce-7afd9f03896d service cinder] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:25:18.923178 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c225b661-23fd-4af8-8cce-7afd9f03896d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:25:18.924999 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-c225b661-23fd-4af8-8cce-7afd9f03896d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:25:18.982559 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:25:19.043149 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c225b661-23fd-4af8-8cce-7afd9f03896d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:25:19.044949 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c225b661-23fd-4af8-8cce-7afd9f03896d service cinder] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:25:19.056584 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1038/2053] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 12:25:18 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 265 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 12:25:19.117601 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc0f1ba-904f-48ac-9641-0faa5b1a22d2 admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:25:19.119426 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-7bc0f1ba-904f-48ac-9641-0faa5b1a22d2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:25:19.192075 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc0f1ba-904f-48ac-9641-0faa5b1a22d2 admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:25:19.197232 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6bb7076-bc8c-419d-8c24-075f7e125bfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:25:19.198055 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bb7076-bc8c-419d-8c24-075f7e125bfb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:25:19.198331 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bb7076-bc8c-419d-8c24-075f7e125bfb admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:25:19.198634 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bb7076-bc8c-419d-8c24-075f7e125bfb admin admin] PATH_INFO: `/v3/projects/3c486beb2a854b8b8c37bcd768e651e8` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:25:19.199772 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6bb7076-bc8c-419d-8c24-075f7e125bfb admin admin] RBAC: Authorizing `identity:delete_project(project_id=3c486beb2a854b8b8c37bcd768e651e8)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:25:19.201713 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6bb7076-bc8c-419d-8c24-075f7e125bfb admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:25:19.254276 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-d6bb7076-bc8c-419d-8c24-075f7e125bfb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3c486beb2a854b8b8c37bcd768e651e8', 'request_id': 'req-d6bb7076-bc8c-419d-8c24-075f7e125bfb'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:25:19.263439 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-d6bb7076-bc8c-419d-8c24-075f7e125bfb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3c486beb2a854b8b8c37bcd768e651e8', 'request_id': 'req-d6bb7076-bc8c-419d-8c24-075f7e125bfb'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:25:19.270707 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-d6bb7076-bc8c-419d-8c24-075f7e125bfb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3c486beb2a854b8b8c37bcd768e651e8', 'request_id': 'req-d6bb7076-bc8c-419d-8c24-075f7e125bfb'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:25:19.283438 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-d6bb7076-bc8c-419d-8c24-075f7e125bfb admin admin] The token cache is being invalidate because project 3c486beb2a854b8b8c37bcd768e651e8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81448) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 12:25:19.283749 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-d6bb7076-bc8c-419d-8c24-075f7e125bfb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:25:19.284665 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-d6bb7076-bc8c-419d-8c24-075f7e125bfb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:25:19.286511 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1016/2054] 10.222.0.15 () {58 vars in 1233 bytes} [Thu Sep 15 12:25:19 2022] DELETE /identity/v3/projects/3c486beb2a854b8b8c37bcd768e651e8 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:25:19.293853 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55acf5fb-2342-40aa-aea0-7efbd98ed6f1 None None] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:25:19.294422 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55acf5fb-2342-40aa-aea0-7efbd98ed6f1 None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:25:19.294779 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55acf5fb-2342-40aa-aea0-7efbd98ed6f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:25:19.318459 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.auth.core [None req-55acf5fb-2342-40aa-aea0-7efbd98ed6f1 None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81449) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:25:19.320104 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-55acf5fb-2342-40aa-aea0-7efbd98ed6f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:25:19.383961 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1039/2055] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 12:25:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:25:19.393897 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ed80e5-33aa-49ef-98ce-4ff6d0c72fd4 None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:25:19.394040 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ed80e5-33aa-49ef-98ce-4ff6d0c72fd4 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:25:19.394316 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ed80e5-33aa-49ef-98ce-4ff6d0c72fd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:25:19.414677 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-b1ed80e5-33aa-49ef-98ce-4ff6d0c72fd4 None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:25:19.415891 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-b1ed80e5-33aa-49ef-98ce-4ff6d0c72fd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:25:19.424429 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1017/2056] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 12:25:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:25:19.429067 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55acf5fb-2342-40aa-aea0-7efbd98ed6f1 None None] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:25:19.439433 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55acf5fb-2342-40aa-aea0-7efbd98ed6f1 None None] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:25:19.444076 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ee42f4-64bb-40c0-91db-cf079e8de709 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:25:19.444684 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ee42f4-64bb-40c0-91db-cf079e8de709 admin admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:25:19.444994 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ee42f4-64bb-40c0-91db-cf079e8de709 admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:25:19.445268 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ee42f4-64bb-40c0-91db-cf079e8de709 admin admin] PATH_INFO: `/v3/domains` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:25:19.445919 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ee42f4-64bb-40c0-91db-cf079e8de709 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81449) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:25:19.446229 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ee42f4-64bb-40c0-91db-cf079e8de709 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:25:19.448344 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ee42f4-64bb-40c0-91db-cf079e8de709 admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:25:19.465382 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1040/2057] 10.222.0.15 () {58 vars in 1153 bytes} [Thu Sep 15 12:25:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:25:19.466503 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ed80e5-33aa-49ef-98ce-4ff6d0c72fd4 None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:25:19.475892 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ed80e5-33aa-49ef-98ce-4ff6d0c72fd4 None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:25:19.481050 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dcabf79-f5c7-4ae4-830a-bea1e3947d99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:25:19.481776 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcabf79-f5c7-4ae4-830a-bea1e3947d99 admin admin] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:25:19.482018 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcabf79-f5c7-4ae4-830a-bea1e3947d99 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:25:19.482271 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcabf79-f5c7-4ae4-830a-bea1e3947d99 admin admin] PATH_INFO: `/v3/projects` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:25:19.482922 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dcabf79-f5c7-4ae4-830a-bea1e3947d99 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:25:19.484844 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dcabf79-f5c7-4ae4-830a-bea1e3947d99 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:25:19.513555 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1018/2058] 10.222.0.15 () {60 vars in 1153 bytes} [Thu Sep 15 12:25:19 2022] POST /identity/v3/projects => generated 371 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 15 12:25:19.518027 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ee42f4-64bb-40c0-91db-cf079e8de709 admin admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:25:19.529089 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ee42f4-64bb-40c0-91db-cf079e8de709 admin admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:25:19.593690 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6794f60a-af14-44e0-8dc7-34cc0ccb967b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:25:19.594308 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6794f60a-af14-44e0-8dc7-34cc0ccb967b admin admin] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:25:19.594579 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6794f60a-af14-44e0-8dc7-34cc0ccb967b admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:25:19.595256 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6794f60a-af14-44e0-8dc7-34cc0ccb967b admin admin] PATH_INFO: `/v3/users` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:25:19.595909 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6794f60a-af14-44e0-8dc7-34cc0ccb967b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:25:19.597969 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6794f60a-af14-44e0-8dc7-34cc0ccb967b admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:25:19.621961 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1041/2059] 10.222.0.15 () {60 vars in 1144 bytes} [Thu Sep 15 12:25:19 2022] POST /identity/v3/users => generated 286 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 12:25:19.626054 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dcabf79-f5c7-4ae4-830a-bea1e3947d99 admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:25:19.638539 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dcabf79-f5c7-4ae4-830a-bea1e3947d99 admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:25:19.643758 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f6aafc-8cad-4d38-8c5f-d4225c796750 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:25:19.644424 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f6aafc-8cad-4d38-8c5f-d4225c796750 admin admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:25:19.644712 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f6aafc-8cad-4d38-8c5f-d4225c796750 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:25:19.645096 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f6aafc-8cad-4d38-8c5f-d4225c796750 admin admin] PATH_INFO: `/v3/roles` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:25:19.645695 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f6aafc-8cad-4d38-8c5f-d4225c796750 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:25:19.647333 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f6aafc-8cad-4d38-8c5f-d4225c796750 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:25:19.659525 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1019/2060] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 12:25:19 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 12:25:19.663827 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6794f60a-af14-44e0-8dc7-34cc0ccb967b admin admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:25:19.674055 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6794f60a-af14-44e0-8dc7-34cc0ccb967b admin admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:25:19.678784 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d4f85c-4153-4e72-aa94-9a5a942bfec6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:25:19.679398 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d4f85c-4153-4e72-aa94-9a5a942bfec6 admin admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:25:19.679678 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d4f85c-4153-4e72-aa94-9a5a942bfec6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:25:19.679954 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d4f85c-4153-4e72-aa94-9a5a942bfec6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:25:19.680783 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d4f85c-4153-4e72-aa94-9a5a942bfec6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:25:19.683118 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d4f85c-4153-4e72-aa94-9a5a942bfec6 admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:25:19.683489 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f6aafc-8cad-4d38-8c5f-d4225c796750 admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:25:19.685333 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-59f6aafc-8cad-4d38-8c5f-d4225c796750 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:25:19.703540 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1042/2061] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 12:25:19 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 15 12:25:19.704634 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d4f85c-4153-4e72-aa94-9a5a942bfec6 admin admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:25:19.718943 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d4f85c-4153-4e72-aa94-9a5a942bfec6 admin admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:25:19.721270 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=88,keystone:DELETE=9,keystone:INSERT=3 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:25:19.725490 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680d9cce-81ce-4ab0-ac6b-b1b60d03291a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:25:19.726334 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680d9cce-81ce-4ab0-ac6b-b1b60d03291a admin admin] REQUEST_METHOD: `PUT` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:25:19.726784 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680d9cce-81ce-4ab0-ac6b-b1b60d03291a admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:25:19.727194 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680d9cce-81ce-4ab0-ac6b-b1b60d03291a admin admin] PATH_INFO: `/v3/projects/175ba2b3abbf44b29d7491b89c73e8a6/users/be06b8c10a5949b3a7a1c31527c63b90/roles/0679d6628a25438d90dc4bdc0401eda2` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:25:19.746860 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680d9cce-81ce-4ab0-ac6b-b1b60d03291a admin admin] RBAC: Authorizing `identity:create_grant(project_id=175ba2b3abbf44b29d7491b89c73e8a6, user_id=be06b8c10a5949b3a7a1c31527c63b90, role_id=0679d6628a25438d90dc4bdc0401eda2)` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:25:19.752948 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680d9cce-81ce-4ab0-ac6b-b1b60d03291a admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:25:19.767442 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f6aafc-8cad-4d38-8c5f-d4225c796750 admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:25:19.772129 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72d62bc-683a-4cf2-a82f-f389e95701e5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8191d3ccb874529b32f8d0026db629d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '540a32ef88764e8c8d9201ae22f6ddaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:25:19.772978 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72d62bc-683a-4cf2-a82f-f389e95701e5 service cinder] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:25:19.773399 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72d62bc-683a-4cf2-a82f-f389e95701e5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:25:19.773792 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72d62bc-683a-4cf2-a82f-f389e95701e5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:25:19.775735 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-e72d62bc-683a-4cf2-a82f-f389e95701e5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:25:19.779231 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1043/2062] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 15 12:25:19 2022] PUT /identity/v3/projects/175ba2b3abbf44b29d7491b89c73e8a6/users/be06b8c10a5949b3a7a1c31527c63b90/roles/0679d6628a25438d90dc4bdc0401eda2 => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 12:25:19.791113 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c43b3f8-5e93-4e2f-84ce-d603bcc202f0 None None] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:25:19.791558 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c43b3f8-5e93-4e2f-84ce-d603bcc202f0 None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:25:19.792086 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c43b3f8-5e93-4e2f-84ce-d603bcc202f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:25:19.816167 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:25:19.825753 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.auth.core [None req-3c43b3f8-5e93-4e2f-84ce-d603bcc202f0 None None] MFA Rules not processed for user `be06b8c10a5949b3a7a1c31527c63b90`. Rule list: `[]` (Enabled: `True`). {{(pid=81449) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:25:19.827018 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-3c43b3f8-5e93-4e2f-84ce-d603bcc202f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:25:19.842860 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e72d62bc-683a-4cf2-a82f-f389e95701e5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:25:19.844588 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e72d62bc-683a-4cf2-a82f-f389e95701e5 service cinder] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:25:19.855326 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1020/2063] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 12:25:19 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 12:25:19.914425 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1044/2064] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 12:25:19 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 12:25:19.921523 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72d62bc-683a-4cf2-a82f-f389e95701e5 service cinder] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:25:19.930207 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72d62bc-683a-4cf2-a82f-f389e95701e5 service cinder] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:25:19.991572 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd68a65-03d2-4bd8-8d2c-5f3f5a296475 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8191d3ccb874529b32f8d0026db629d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '540a32ef88764e8c8d9201ae22f6ddaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:25:19.992316 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd68a65-03d2-4bd8-8d2c-5f3f5a296475 service cinder] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:25:19.992642 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd68a65-03d2-4bd8-8d2c-5f3f5a296475 service cinder] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:25:19.992984 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd68a65-03d2-4bd8-8d2c-5f3f5a296475 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:25:20.002684 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd68a65-03d2-4bd8-8d2c-5f3f5a296475 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:25:20.004037 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd68a65-03d2-4bd8-8d2c-5f3f5a296475 service cinder] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:25:20.013339 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1021/2065] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 12:25:19 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 12:25:20.035022 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c43b3f8-5e93-4e2f-84ce-d603bcc202f0 None None] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:25:20.097574 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c43b3f8-5e93-4e2f-84ce-d603bcc202f0 None None] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:25:20.102152 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8ac779-d65d-4d89-83b1-9037d5a98a86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:25:20.102780 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8ac779-d65d-4d89-83b1-9037d5a98a86 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:25:20.103069 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8ac779-d65d-4d89-83b1-9037d5a98a86 admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:25:20.103346 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8ac779-d65d-4d89-83b1-9037d5a98a86 admin admin] PATH_INFO: `/v3/users/be06b8c10a5949b3a7a1c31527c63b90` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:25:20.104482 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8ac779-d65d-4d89-83b1-9037d5a98a86 admin admin] RBAC: Authorizing `identity:delete_user(user_id=be06b8c10a5949b3a7a1c31527c63b90)` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:25:20.106433 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8ac779-d65d-4d89-83b1-9037d5a98a86 admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:25:20.155327 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-fd8ac779-d65d-4d89-83b1-9037d5a98a86 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'be06b8c10a5949b3a7a1c31527c63b90'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:25:20.162981 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-fd8ac779-d65d-4d89-83b1-9037d5a98a86 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'be06b8c10a5949b3a7a1c31527c63b90'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:25:20.176932 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-fd8ac779-d65d-4d89-83b1-9037d5a98a86 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'be06b8c10a5949b3a7a1c31527c63b90'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:25:20.177647 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-fd8ac779-d65d-4d89-83b1-9037d5a98a86 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'be06b8c10a5949b3a7a1c31527c63b90'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:25:20.185576 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-fd8ac779-d65d-4d89-83b1-9037d5a98a86 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'be06b8c10a5949b3a7a1c31527c63b90'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:25:20.191819 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1045/2066] 10.222.0.15 () {58 vars in 1224 bytes} [Thu Sep 15 12:25:20 2022] DELETE /identity/v3/users/be06b8c10a5949b3a7a1c31527c63b90 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:25:20.197472 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869c9568-faf9-4106-926a-7b1f480a1264 None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:25:20.198504 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869c9568-faf9-4106-926a-7b1f480a1264 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:25:20.199001 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869c9568-faf9-4106-926a-7b1f480a1264 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:25:20.219083 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-869c9568-faf9-4106-926a-7b1f480a1264 None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:25:20.220556 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-869c9568-faf9-4106-926a-7b1f480a1264 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:25:20.289704 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1022/2067] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 12:25:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:25:20.301052 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8ac779-d65d-4d89-83b1-9037d5a98a86 admin admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:25:20.303096 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-fd8ac779-d65d-4d89-83b1-9037d5a98a86 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:25:20.328359 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:25:20.371647 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8ac779-d65d-4d89-83b1-9037d5a98a86 admin admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:25:20.376466 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3913c8-d9b1-42dc-be79-2492f70efa56 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfbb4d6809064f9cb9e06626ed7d6814', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '540a32ef88764e8c8d9201ae22f6ddaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:25:20.377108 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3913c8-d9b1-42dc-be79-2492f70efa56 service neutron] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:25:20.377391 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3913c8-d9b1-42dc-be79-2492f70efa56 service neutron] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:25:20.377666 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3913c8-d9b1-42dc-be79-2492f70efa56 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:25:20.388114 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3913c8-d9b1-42dc-be79-2492f70efa56 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:25:20.389680 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3913c8-d9b1-42dc-be79-2492f70efa56 service neutron] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:25:20.399519 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1046/2068] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 12:25:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 12:25:20.674085 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869c9568-faf9-4106-926a-7b1f480a1264 None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:25:20.675536 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-869c9568-faf9-4106-926a-7b1f480a1264 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:25:20.687790 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869c9568-faf9-4106-926a-7b1f480a1264 None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:25:20.692622 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5dcda68-db4a-4051-8991-761688ecea2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:25:20.693410 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5dcda68-db4a-4051-8991-761688ecea2d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:25:20.693692 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5dcda68-db4a-4051-8991-761688ecea2d admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:25:20.693964 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5dcda68-db4a-4051-8991-761688ecea2d admin admin] PATH_INFO: `/v3/projects/175ba2b3abbf44b29d7491b89c73e8a6` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:25:20.695084 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5dcda68-db4a-4051-8991-761688ecea2d admin admin] RBAC: Authorizing `identity:delete_project(project_id=175ba2b3abbf44b29d7491b89c73e8a6)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:25:20.697471 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5dcda68-db4a-4051-8991-761688ecea2d admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:25:20.713831 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:25:20.745088 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-a5dcda68-db4a-4051-8991-761688ecea2d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '175ba2b3abbf44b29d7491b89c73e8a6', 'request_id': 'req-a5dcda68-db4a-4051-8991-761688ecea2d'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:25:20.753445 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-a5dcda68-db4a-4051-8991-761688ecea2d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '175ba2b3abbf44b29d7491b89c73e8a6', 'request_id': 'req-a5dcda68-db4a-4051-8991-761688ecea2d'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:25:20.759849 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-a5dcda68-db4a-4051-8991-761688ecea2d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '175ba2b3abbf44b29d7491b89c73e8a6', 'request_id': 'req-a5dcda68-db4a-4051-8991-761688ecea2d'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:25:20.771094 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-a5dcda68-db4a-4051-8991-761688ecea2d admin admin] The token cache is being invalidate because project 175ba2b3abbf44b29d7491b89c73e8a6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81448) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 12:25:20.771386 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-a5dcda68-db4a-4051-8991-761688ecea2d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:25:20.772043 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-a5dcda68-db4a-4051-8991-761688ecea2d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:25:20.773599 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1023/2069] 10.222.0.15 () {58 vars in 1233 bytes} [Thu Sep 15 12:25:20 2022] DELETE /identity/v3/projects/175ba2b3abbf44b29d7491b89c73e8a6 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:25:20.779064 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ace927-ef17-41b2-aaa9-12fff06956c6 None None] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:25:20.779350 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ace927-ef17-41b2-aaa9-12fff06956c6 None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:25:20.779624 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ace927-ef17-41b2-aaa9-12fff06956c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:25:20.799170 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.auth.core [None req-a6ace927-ef17-41b2-aaa9-12fff06956c6 None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81449) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:25:20.800288 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-a6ace927-ef17-41b2-aaa9-12fff06956c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:25:20.860983 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1047/2070] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 12:25:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 12:25:20.868712 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c59b63-0bf2-48b5-968a-b57ee4ebc2ad None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:25:20.869025 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c59b63-0bf2-48b5-968a-b57ee4ebc2ad None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:25:20.869300 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c59b63-0bf2-48b5-968a-b57ee4ebc2ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:25:20.889218 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-77c59b63-0bf2-48b5-968a-b57ee4ebc2ad None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:25:20.890446 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-77c59b63-0bf2-48b5-968a-b57ee4ebc2ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:25:20.898354 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1024/2071] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 12:25:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:25:20.902671 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ace927-ef17-41b2-aaa9-12fff06956c6 None None] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:25:20.913005 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ace927-ef17-41b2-aaa9-12fff06956c6 None None] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:25:20.917539 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88553be-fa04-42e0-8038-12ff786932c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:25:20.918161 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88553be-fa04-42e0-8038-12ff786932c3 admin admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:25:20.918439 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88553be-fa04-42e0-8038-12ff786932c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:25:20.918706 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88553be-fa04-42e0-8038-12ff786932c3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:25:20.919344 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88553be-fa04-42e0-8038-12ff786932c3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81449) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:25:20.919644 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88553be-fa04-42e0-8038-12ff786932c3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:25:20.921426 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88553be-fa04-42e0-8038-12ff786932c3 admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:25:20.933305 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1048/2072] 10.222.0.15 () {58 vars in 1153 bytes} [Thu Sep 15 12:25:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:25:20.937701 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c59b63-0bf2-48b5-968a-b57ee4ebc2ad None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:25:20.946792 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c59b63-0bf2-48b5-968a-b57ee4ebc2ad None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:25:20.951886 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3689a5cf-aa29-494d-a70a-e5ab3b9fb0f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:25:20.952479 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3689a5cf-aa29-494d-a70a-e5ab3b9fb0f9 admin admin] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:25:20.952781 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3689a5cf-aa29-494d-a70a-e5ab3b9fb0f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:25:20.953073 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3689a5cf-aa29-494d-a70a-e5ab3b9fb0f9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:25:20.953710 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3689a5cf-aa29-494d-a70a-e5ab3b9fb0f9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:25:20.955598 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3689a5cf-aa29-494d-a70a-e5ab3b9fb0f9 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:25:20.984557 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1025/2073] 10.222.0.15 () {60 vars in 1153 bytes} [Thu Sep 15 12:25:20 2022] POST /identity/v3/projects => generated 385 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 12:25:20.988416 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88553be-fa04-42e0-8038-12ff786932c3 admin admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:25:21.002035 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88553be-fa04-42e0-8038-12ff786932c3 admin admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:25:21.079875 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa622764-64a7-4090-a266-0df145a5ec56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:25:21.080512 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa622764-64a7-4090-a266-0df145a5ec56 admin admin] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:25:21.080820 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa622764-64a7-4090-a266-0df145a5ec56 admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:25:21.081095 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa622764-64a7-4090-a266-0df145a5ec56 admin admin] PATH_INFO: `/v3/users` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:25:21.081740 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa622764-64a7-4090-a266-0df145a5ec56 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:25:21.083747 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa622764-64a7-4090-a266-0df145a5ec56 admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:25:21.117005 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1049/2074] 10.222.0.15 () {60 vars in 1144 bytes} [Thu Sep 15 12:25:20 2022] POST /identity/v3/users => generated 293 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 15 12:25:21.118193 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3689a5cf-aa29-494d-a70a-e5ab3b9fb0f9 admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:25:21.136880 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3689a5cf-aa29-494d-a70a-e5ab3b9fb0f9 admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:25:21.141943 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02aebe88-d504-4ecf-a200-8b21c2ba5248 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:25:21.142645 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02aebe88-d504-4ecf-a200-8b21c2ba5248 admin admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:25:21.142930 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02aebe88-d504-4ecf-a200-8b21c2ba5248 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:25:21.143233 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02aebe88-d504-4ecf-a200-8b21c2ba5248 admin admin] PATH_INFO: `/v3/roles` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:25:21.144000 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02aebe88-d504-4ecf-a200-8b21c2ba5248 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:25:21.145621 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02aebe88-d504-4ecf-a200-8b21c2ba5248 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:25:21.157348 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1026/2075] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 12:25:21 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 12:25:21.161069 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa622764-64a7-4090-a266-0df145a5ec56 admin admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:25:21.174315 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa622764-64a7-4090-a266-0df145a5ec56 admin admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:25:21.180322 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3e820d-c728-4ed8-94e5-01c65de85b5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:25:21.181541 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3e820d-c728-4ed8-94e5-01c65de85b5c admin admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:25:21.185649 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3e820d-c728-4ed8-94e5-01c65de85b5c admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:25:21.185993 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3e820d-c728-4ed8-94e5-01c65de85b5c admin admin] PATH_INFO: `/v3/roles` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:25:21.186815 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3e820d-c728-4ed8-94e5-01c65de85b5c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:25:21.188438 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3e820d-c728-4ed8-94e5-01c65de85b5c admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:25:21.202932 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1050/2076] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 12:25:21 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 12:25:21.207684 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02aebe88-d504-4ecf-a200-8b21c2ba5248 admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:25:21.223632 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02aebe88-d504-4ecf-a200-8b21c2ba5248 admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:25:21.229550 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c1441e-7656-43f7-bc9c-e531d2a09798 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:25:21.232062 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c1441e-7656-43f7-bc9c-e531d2a09798 admin admin] REQUEST_METHOD: `PUT` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:25:21.232406 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c1441e-7656-43f7-bc9c-e531d2a09798 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:25:21.232691 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c1441e-7656-43f7-bc9c-e531d2a09798 admin admin] PATH_INFO: `/v3/projects/506b60c4fe2b4e3faa9c5ec182b7fcf9/users/1e57e16f2b2d47d38f7be8bb48486c2f/roles/0679d6628a25438d90dc4bdc0401eda2` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:25:21.254309 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c1441e-7656-43f7-bc9c-e531d2a09798 admin admin] RBAC: Authorizing `identity:create_grant(project_id=506b60c4fe2b4e3faa9c5ec182b7fcf9, user_id=1e57e16f2b2d47d38f7be8bb48486c2f, role_id=0679d6628a25438d90dc4bdc0401eda2)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:25:21.258022 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c1441e-7656-43f7-bc9c-e531d2a09798 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:25:21.276291 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1027/2077] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 15 12:25:21 2022] PUT /identity/v3/projects/506b60c4fe2b4e3faa9c5ec182b7fcf9/users/1e57e16f2b2d47d38f7be8bb48486c2f/roles/0679d6628a25438d90dc4bdc0401eda2 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:25:21.289931 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc14e854-bc22-4a2f-b691-b1199164cc4a None None] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:25:21.290386 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc14e854-bc22-4a2f-b691-b1199164cc4a None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:25:21.290838 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc14e854-bc22-4a2f-b691-b1199164cc4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:25:21.341386 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.auth.core [None req-dc14e854-bc22-4a2f-b691-b1199164cc4a None None] MFA Rules not processed for user `1e57e16f2b2d47d38f7be8bb48486c2f`. Rule list: `[]` (Enabled: `True`). {{(pid=81449) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:25:21.342762 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-dc14e854-bc22-4a2f-b691-b1199164cc4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:25:21.393770 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:25:21.502125 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1051/2078] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 12:25:21 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:25:21.511457 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c1441e-7656-43f7-bc9c-e531d2a09798 admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:25:21.514139 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-b5c1441e-7656-43f7-bc9c-e531d2a09798 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:25:21.552105 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc14e854-bc22-4a2f-b691-b1199164cc4a None None] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:25:21.553556 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-dc14e854-bc22-4a2f-b691-b1199164cc4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:25:21.595705 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c1441e-7656-43f7-bc9c-e531d2a09798 admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:25:21.599411 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a459fb57-cac5-4206-9ca1-030908db0cba service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8191d3ccb874529b32f8d0026db629d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '540a32ef88764e8c8d9201ae22f6ddaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:25:21.600149 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a459fb57-cac5-4206-9ca1-030908db0cba service cinder] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:25:21.600425 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a459fb57-cac5-4206-9ca1-030908db0cba service cinder] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:25:21.600689 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a459fb57-cac5-4206-9ca1-030908db0cba service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:25:21.610333 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a459fb57-cac5-4206-9ca1-030908db0cba service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:25:21.611915 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a459fb57-cac5-4206-9ca1-030908db0cba service cinder] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:25:21.621790 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1028/2079] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 12:25:21 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 12:25:21.636135 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc14e854-bc22-4a2f-b691-b1199164cc4a None None] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:25:21.641116 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b52b4ae-0c42-4426-998f-b31023aa3da4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:25:21.641941 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b52b4ae-0c42-4426-998f-b31023aa3da4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:25:21.642266 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b52b4ae-0c42-4426-998f-b31023aa3da4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:25:21.642590 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b52b4ae-0c42-4426-998f-b31023aa3da4 admin admin] PATH_INFO: `/v3/users/754a45f1f5ab4e80a7b9f19b39830bbf` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:25:21.643767 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b52b4ae-0c42-4426-998f-b31023aa3da4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=754a45f1f5ab4e80a7b9f19b39830bbf)` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:25:21.645949 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b52b4ae-0c42-4426-998f-b31023aa3da4 admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:25:21.732743 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-8b52b4ae-0c42-4426-998f-b31023aa3da4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '754a45f1f5ab4e80a7b9f19b39830bbf'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:25:21.744691 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-8b52b4ae-0c42-4426-998f-b31023aa3da4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '754a45f1f5ab4e80a7b9f19b39830bbf'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:25:21.763315 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-8b52b4ae-0c42-4426-998f-b31023aa3da4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '754a45f1f5ab4e80a7b9f19b39830bbf'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:25:21.764488 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-8b52b4ae-0c42-4426-998f-b31023aa3da4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '754a45f1f5ab4e80a7b9f19b39830bbf'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:25:21.774733 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-8b52b4ae-0c42-4426-998f-b31023aa3da4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '754a45f1f5ab4e80a7b9f19b39830bbf'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:25:21.783844 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1052/2080] 10.222.0.15 () {58 vars in 1224 bytes} [Thu Sep 15 12:25:21 2022] DELETE /identity/v3/users/754a45f1f5ab4e80a7b9f19b39830bbf => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 12:25:21.788580 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c777ff9-ffa3-4377-9992-b3bc75bc5e12 None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:25:21.789156 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c777ff9-ffa3-4377-9992-b3bc75bc5e12 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:25:21.789762 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c777ff9-ffa3-4377-9992-b3bc75bc5e12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:25:21.821744 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-2c777ff9-ffa3-4377-9992-b3bc75bc5e12 None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:25:21.823756 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-2c777ff9-ffa3-4377-9992-b3bc75bc5e12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:25:21.890455 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:25:21.951685 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1029/2081] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 12:25:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 12:25:21.962546 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b52b4ae-0c42-4426-998f-b31023aa3da4 admin admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:25:21.964888 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-8b52b4ae-0c42-4426-998f-b31023aa3da4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:25:22.028350 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:25:22.048213 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b52b4ae-0c42-4426-998f-b31023aa3da4 admin admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:25:22.052811 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e289f9d-baa7-4382-b237-91e84a66b54f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfbb4d6809064f9cb9e06626ed7d6814', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '540a32ef88764e8c8d9201ae22f6ddaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:25:22.053509 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e289f9d-baa7-4382-b237-91e84a66b54f service neutron] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:25:22.053788 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e289f9d-baa7-4382-b237-91e84a66b54f service neutron] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:25:22.054076 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e289f9d-baa7-4382-b237-91e84a66b54f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:25:22.064445 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e289f9d-baa7-4382-b237-91e84a66b54f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:25:22.065871 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e289f9d-baa7-4382-b237-91e84a66b54f service neutron] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:25:22.078501 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1053/2082] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 12:25:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 12:25:22.377404 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c777ff9-ffa3-4377-9992-b3bc75bc5e12 None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:25:22.378796 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-2c777ff9-ffa3-4377-9992-b3bc75bc5e12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:25:22.394840 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c777ff9-ffa3-4377-9992-b3bc75bc5e12 None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:25:22.400261 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054122dc-ff94-40bc-96f4-efe38cbb0122 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:25:22.401057 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054122dc-ff94-40bc-96f4-efe38cbb0122 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:25:22.401317 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054122dc-ff94-40bc-96f4-efe38cbb0122 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:25:22.401585 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054122dc-ff94-40bc-96f4-efe38cbb0122 admin admin] PATH_INFO: `/v3/projects/c1f0f49f343342b0ab454757d2547f6d` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:25:22.402733 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054122dc-ff94-40bc-96f4-efe38cbb0122 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c1f0f49f343342b0ab454757d2547f6d)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:25:22.404639 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054122dc-ff94-40bc-96f4-efe38cbb0122 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:25:22.458039 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-054122dc-ff94-40bc-96f4-efe38cbb0122 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c1f0f49f343342b0ab454757d2547f6d', 'request_id': 'req-054122dc-ff94-40bc-96f4-efe38cbb0122'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:25:22.467276 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-054122dc-ff94-40bc-96f4-efe38cbb0122 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c1f0f49f343342b0ab454757d2547f6d', 'request_id': 'req-054122dc-ff94-40bc-96f4-efe38cbb0122'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:25:22.474146 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-054122dc-ff94-40bc-96f4-efe38cbb0122 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c1f0f49f343342b0ab454757d2547f6d', 'request_id': 'req-054122dc-ff94-40bc-96f4-efe38cbb0122'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:25:22.487782 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-054122dc-ff94-40bc-96f4-efe38cbb0122 admin admin] The token cache is being invalidate because project c1f0f49f343342b0ab454757d2547f6d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81448) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 12:25:22.488057 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-054122dc-ff94-40bc-96f4-efe38cbb0122 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:25:22.488887 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-054122dc-ff94-40bc-96f4-efe38cbb0122 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:25:22.490603 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1030/2083] 10.222.0.15 () {58 vars in 1233 bytes} [Thu Sep 15 12:25:22 2022] DELETE /identity/v3/projects/c1f0f49f343342b0ab454757d2547f6d => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:25:22.495088 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e289f9d-baa7-4382-b237-91e84a66b54f service neutron] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:25:22.514995 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e289f9d-baa7-4382-b237-91e84a66b54f service neutron] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:25:22.573571 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b4851d-590a-44ca-ae5e-869cd74f9c31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:25:22.574298 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b4851d-590a-44ca-ae5e-869cd74f9c31 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:25:22.574583 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b4851d-590a-44ca-ae5e-869cd74f9c31 admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:25:22.574858 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b4851d-590a-44ca-ae5e-869cd74f9c31 admin admin] PATH_INFO: `/v3/users/7e07985e44e84a459dcf526e8740fa82` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:25:22.575975 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b4851d-590a-44ca-ae5e-869cd74f9c31 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7e07985e44e84a459dcf526e8740fa82)` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:25:22.577936 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b4851d-590a-44ca-ae5e-869cd74f9c31 admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:25:22.653091 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-c4b4851d-590a-44ca-ae5e-869cd74f9c31 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7e07985e44e84a459dcf526e8740fa82'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:25:22.674921 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-c4b4851d-590a-44ca-ae5e-869cd74f9c31 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7e07985e44e84a459dcf526e8740fa82'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:25:22.682517 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-c4b4851d-590a-44ca-ae5e-869cd74f9c31 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7e07985e44e84a459dcf526e8740fa82'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:25:22.683754 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-c4b4851d-590a-44ca-ae5e-869cd74f9c31 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7e07985e44e84a459dcf526e8740fa82'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:25:22.692405 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-c4b4851d-590a-44ca-ae5e-869cd74f9c31 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7e07985e44e84a459dcf526e8740fa82'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:25:22.706912 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1054/2084] 10.222.0.15 () {58 vars in 1224 bytes} [Thu Sep 15 12:25:22 2022] DELETE /identity/v3/users/7e07985e44e84a459dcf526e8740fa82 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 12:25:23.039994 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054122dc-ff94-40bc-96f4-efe38cbb0122 admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:25:23.041486 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-054122dc-ff94-40bc-96f4-efe38cbb0122 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:25:23.055103 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054122dc-ff94-40bc-96f4-efe38cbb0122 admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:25:23.127305 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e7351c-0cbf-45f7-a935-a73e6315bb84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:25:23.128035 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e7351c-0cbf-45f7-a935-a73e6315bb84 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:25:23.128334 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e7351c-0cbf-45f7-a935-a73e6315bb84 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:25:23.128610 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e7351c-0cbf-45f7-a935-a73e6315bb84 admin admin] PATH_INFO: `/v3/projects/3007eb79cc2840e988a378f231c79fe3` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:25:23.129801 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e7351c-0cbf-45f7-a935-a73e6315bb84 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3007eb79cc2840e988a378f231c79fe3)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:25:23.131869 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e7351c-0cbf-45f7-a935-a73e6315bb84 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:25:23.204964 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-d8e7351c-0cbf-45f7-a935-a73e6315bb84 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3007eb79cc2840e988a378f231c79fe3', 'request_id': 'req-d8e7351c-0cbf-45f7-a935-a73e6315bb84'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:25:23.213685 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-d8e7351c-0cbf-45f7-a935-a73e6315bb84 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3007eb79cc2840e988a378f231c79fe3', 'request_id': 'req-d8e7351c-0cbf-45f7-a935-a73e6315bb84'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:25:23.221236 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-d8e7351c-0cbf-45f7-a935-a73e6315bb84 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3007eb79cc2840e988a378f231c79fe3', 'request_id': 'req-d8e7351c-0cbf-45f7-a935-a73e6315bb84'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:25:23.227687 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:25:23.236345 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-d8e7351c-0cbf-45f7-a935-a73e6315bb84 admin admin] The token cache is being invalidate because project 3007eb79cc2840e988a378f231c79fe3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81448) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 12:25:23.236345 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-d8e7351c-0cbf-45f7-a935-a73e6315bb84 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:25:23.237514 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-d8e7351c-0cbf-45f7-a935-a73e6315bb84 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:25:23.239272 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1031/2085] 10.222.0.15 () {58 vars in 1233 bytes} [Thu Sep 15 12:25:23 2022] DELETE /identity/v3/projects/3007eb79cc2840e988a378f231c79fe3 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:25:23.244064 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b6c657-6737-4de0-9fc2-73b8e679e34c None None] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:25:23.244927 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b6c657-6737-4de0-9fc2-73b8e679e34c None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:25:23.245225 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b6c657-6737-4de0-9fc2-73b8e679e34c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:25:23.265545 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.auth.core [None req-33b6c657-6737-4de0-9fc2-73b8e679e34c None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81449) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:25:23.266798 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-33b6c657-6737-4de0-9fc2-73b8e679e34c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:25:23.333751 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:25:23.346929 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1055/2086] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 12:25:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:25:23.354743 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbd07e9-8497-4530-a654-7a0d5599b3e6 None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:25:23.355031 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbd07e9-8497-4530-a654-7a0d5599b3e6 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:25:23.355301 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbd07e9-8497-4530-a654-7a0d5599b3e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:25:23.376318 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-cbbd07e9-8497-4530-a654-7a0d5599b3e6 None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:25:23.377408 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-cbbd07e9-8497-4530-a654-7a0d5599b3e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:25:23.385344 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1032/2087] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 12:25:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:25:23.389692 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b6c657-6737-4de0-9fc2-73b8e679e34c None None] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:25:23.399355 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b6c657-6737-4de0-9fc2-73b8e679e34c None None] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:25:23.403871 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa3cdef-518b-4a8e-a636-c9f8189da19e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:25:23.404458 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa3cdef-518b-4a8e-a636-c9f8189da19e admin admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:25:23.404734 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa3cdef-518b-4a8e-a636-c9f8189da19e admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:25:23.405029 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa3cdef-518b-4a8e-a636-c9f8189da19e admin admin] PATH_INFO: `/v3/domains` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:25:23.405664 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa3cdef-518b-4a8e-a636-c9f8189da19e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81449) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:25:23.405960 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa3cdef-518b-4a8e-a636-c9f8189da19e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:25:23.407538 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa3cdef-518b-4a8e-a636-c9f8189da19e admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:25:23.419363 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1056/2088] 10.222.0.15 () {58 vars in 1153 bytes} [Thu Sep 15 12:25:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:25:23.423649 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbd07e9-8497-4530-a654-7a0d5599b3e6 None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:25:23.433717 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbd07e9-8497-4530-a654-7a0d5599b3e6 None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:25:23.438277 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf14007-0e6d-4315-bf4b-07bf005243fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:25:23.438869 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf14007-0e6d-4315-bf4b-07bf005243fe admin admin] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:25:23.439151 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf14007-0e6d-4315-bf4b-07bf005243fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:25:23.439421 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf14007-0e6d-4315-bf4b-07bf005243fe admin admin] PATH_INFO: `/v3/projects` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:25:23.440045 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf14007-0e6d-4315-bf4b-07bf005243fe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:25:23.441965 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf14007-0e6d-4315-bf4b-07bf005243fe admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:25:23.468639 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1033/2089] 10.222.0.15 () {60 vars in 1153 bytes} [Thu Sep 15 12:25:23 2022] POST /identity/v3/projects => generated 365 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 12:25:23.472998 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa3cdef-518b-4a8e-a636-c9f8189da19e admin admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:25:23.483325 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa3cdef-518b-4a8e-a636-c9f8189da19e admin admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:25:23.541966 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebe9453-056c-4991-bd48-27f0edcfea6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:25:23.542577 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebe9453-056c-4991-bd48-27f0edcfea6b admin admin] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:25:23.542869 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebe9453-056c-4991-bd48-27f0edcfea6b admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:25:23.543122 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebe9453-056c-4991-bd48-27f0edcfea6b admin admin] PATH_INFO: `/v3/users` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:25:23.543743 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ebe9453-056c-4991-bd48-27f0edcfea6b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:25:23.545660 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ebe9453-056c-4991-bd48-27f0edcfea6b admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:25:23.566628 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1057/2090] 10.222.0.15 () {60 vars in 1144 bytes} [Thu Sep 15 12:25:23 2022] POST /identity/v3/users => generated 283 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 12:25:23.570721 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf14007-0e6d-4315-bf4b-07bf005243fe admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:25:23.581034 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf14007-0e6d-4315-bf4b-07bf005243fe admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:25:23.585592 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60c9289-6670-4952-b90a-c5df13f9c116 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:25:23.586196 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60c9289-6670-4952-b90a-c5df13f9c116 admin admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:25:23.586477 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60c9289-6670-4952-b90a-c5df13f9c116 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:25:23.586748 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60c9289-6670-4952-b90a-c5df13f9c116 admin admin] PATH_INFO: `/v3/roles` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:25:23.587412 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f60c9289-6670-4952-b90a-c5df13f9c116 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:25:23.589560 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f60c9289-6670-4952-b90a-c5df13f9c116 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:25:23.599690 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1034/2091] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 12:25:23 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 12:25:23.603457 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebe9453-056c-4991-bd48-27f0edcfea6b admin admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:25:23.614022 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebe9453-056c-4991-bd48-27f0edcfea6b admin admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:25:23.618585 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc91238-3361-4f13-8d67-74004cfcb6be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:25:23.619179 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc91238-3361-4f13-8d67-74004cfcb6be admin admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:25:23.619482 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc91238-3361-4f13-8d67-74004cfcb6be admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:25:23.619755 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc91238-3361-4f13-8d67-74004cfcb6be admin admin] PATH_INFO: `/v3/roles` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:25:23.620414 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc91238-3361-4f13-8d67-74004cfcb6be admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:25:23.622045 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc91238-3361-4f13-8d67-74004cfcb6be admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:25:23.632409 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1058/2092] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 12:25:23 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 12:25:23.636639 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60c9289-6670-4952-b90a-c5df13f9c116 admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:25:23.646773 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60c9289-6670-4952-b90a-c5df13f9c116 admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:25:23.652207 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d460cc41-a32a-4cb9-8d1d-281099b6d505 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:25:23.652898 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d460cc41-a32a-4cb9-8d1d-281099b6d505 admin admin] REQUEST_METHOD: `PUT` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:25:23.653225 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d460cc41-a32a-4cb9-8d1d-281099b6d505 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:25:23.653573 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d460cc41-a32a-4cb9-8d1d-281099b6d505 admin admin] PATH_INFO: `/v3/projects/3d4e37afdac74a26a14355663bad78ea/users/de293cfdebde41b089afb0cb3ed061e7/roles/0679d6628a25438d90dc4bdc0401eda2` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:25:23.668717 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d460cc41-a32a-4cb9-8d1d-281099b6d505 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3d4e37afdac74a26a14355663bad78ea, user_id=de293cfdebde41b089afb0cb3ed061e7, role_id=0679d6628a25438d90dc4bdc0401eda2)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:25:23.671414 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d460cc41-a32a-4cb9-8d1d-281099b6d505 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:25:23.683444 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1035/2093] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 15 12:25:23 2022] PUT /identity/v3/projects/3d4e37afdac74a26a14355663bad78ea/users/de293cfdebde41b089afb0cb3ed061e7/roles/0679d6628a25438d90dc4bdc0401eda2 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:25:23.693661 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7ad61b-dc9c-403e-9081-1ac8e6a5a6f6 None None] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:25:23.693947 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7ad61b-dc9c-403e-9081-1ac8e6a5a6f6 None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:25:23.694219 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7ad61b-dc9c-403e-9081-1ac8e6a5a6f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:25:23.721640 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.auth.core [None req-bd7ad61b-dc9c-403e-9081-1ac8e6a5a6f6 None None] MFA Rules not processed for user `de293cfdebde41b089afb0cb3ed061e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81449) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:25:23.722834 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-bd7ad61b-dc9c-403e-9081-1ac8e6a5a6f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:25:23.803829 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1059/2094] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 12:25:23 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:25:23.808353 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d460cc41-a32a-4cb9-8d1d-281099b6d505 admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:25:23.871746 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d460cc41-a32a-4cb9-8d1d-281099b6d505 admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:25:23.878051 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c6f77f-92fd-4783-b410-e2513e92917b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:25:23.878672 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c6f77f-92fd-4783-b410-e2513e92917b admin admin] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:25:23.878951 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c6f77f-92fd-4783-b410-e2513e92917b admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:25:23.879220 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c6f77f-92fd-4783-b410-e2513e92917b admin admin] PATH_INFO: `/v3/projects` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:25:23.879862 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c6f77f-92fd-4783-b410-e2513e92917b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:25:23.881928 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c6f77f-92fd-4783-b410-e2513e92917b admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:25:23.914346 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1036/2095] 10.222.0.15 () {60 vars in 1153 bytes} [Thu Sep 15 12:25:23 2022] POST /identity/v3/projects => generated 365 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 12:25:23.915461 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7ad61b-dc9c-403e-9081-1ac8e6a5a6f6 None None] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:25:23.929823 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7ad61b-dc9c-403e-9081-1ac8e6a5a6f6 None None] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:25:23.990643 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:25:24.007213 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67f858b-f637-4b36-8ed1-44e6d2b33310 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:25:24.008476 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67f858b-f637-4b36-8ed1-44e6d2b33310 admin admin] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:25:24.008858 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67f858b-f637-4b36-8ed1-44e6d2b33310 admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:25:24.009197 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67f858b-f637-4b36-8ed1-44e6d2b33310 admin admin] PATH_INFO: `/v3/users` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:25:24.009952 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67f858b-f637-4b36-8ed1-44e6d2b33310 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:25:24.011955 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67f858b-f637-4b36-8ed1-44e6d2b33310 admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:25:24.039298 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1060/2096] 10.222.0.15 () {60 vars in 1144 bytes} [Thu Sep 15 12:25:23 2022] POST /identity/v3/users => generated 289 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 15 12:25:24.046398 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c6f77f-92fd-4783-b410-e2513e92917b admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:25:24.048986 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67f858b-f637-4b36-8ed1-44e6d2b33310 admin admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:25:24.050684 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-a67f858b-f637-4b36-8ed1-44e6d2b33310 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:25:24.058089 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c6f77f-92fd-4783-b410-e2513e92917b admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:25:24.065646 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67f858b-f637-4b36-8ed1-44e6d2b33310 admin admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:25:24.066304 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64600522-eada-49b3-aec2-6dcbe7898694 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:25:24.067013 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64600522-eada-49b3-aec2-6dcbe7898694 admin admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:25:24.067404 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64600522-eada-49b3-aec2-6dcbe7898694 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:25:24.067804 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64600522-eada-49b3-aec2-6dcbe7898694 admin admin] PATH_INFO: `/v3/roles` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:25:24.068462 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64600522-eada-49b3-aec2-6dcbe7898694 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:25:24.070366 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64600522-eada-49b3-aec2-6dcbe7898694 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:25:24.070472 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040c71ab-86bd-42eb-bc47-1f75e09b527b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:25:24.071018 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040c71ab-86bd-42eb-bc47-1f75e09b527b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:25:24.071298 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040c71ab-86bd-42eb-bc47-1f75e09b527b admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:25:24.071566 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040c71ab-86bd-42eb-bc47-1f75e09b527b admin admin] PATH_INFO: `/v3/users/be01f210b2d643cb9678f4cf1802c23e` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:25:24.072926 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040c71ab-86bd-42eb-bc47-1f75e09b527b admin admin] RBAC: Authorizing `identity:delete_user(user_id=be01f210b2d643cb9678f4cf1802c23e)` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:25:24.076205 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040c71ab-86bd-42eb-bc47-1f75e09b527b admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:25:24.084867 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1037/2097] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 12:25:24 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 12:25:24.095481 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64600522-eada-49b3-aec2-6dcbe7898694 admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:25:24.106855 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64600522-eada-49b3-aec2-6dcbe7898694 admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:25:24.111805 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fbfb72b-cb91-41a2-9710-e9939bd6e2e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:25:24.112514 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbfb72b-cb91-41a2-9710-e9939bd6e2e1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:25:24.116986 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbfb72b-cb91-41a2-9710-e9939bd6e2e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:25:24.117898 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fbfb72b-cb91-41a2-9710-e9939bd6e2e1 admin admin] PATH_INFO: `/v3/projects/f23572583527499b9aecb3ee291c60cc/users/bcf49aeb19034e4daf5c5eea501eb727/roles/9c769b5c82884766b2c510b1f0f1364c` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:25:24.133197 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-040c71ab-86bd-42eb-bc47-1f75e09b527b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'be01f210b2d643cb9678f4cf1802c23e'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:25:24.141133 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-040c71ab-86bd-42eb-bc47-1f75e09b527b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'be01f210b2d643cb9678f4cf1802c23e'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:25:24.143755 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fbfb72b-cb91-41a2-9710-e9939bd6e2e1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f23572583527499b9aecb3ee291c60cc, user_id=bcf49aeb19034e4daf5c5eea501eb727, role_id=9c769b5c82884766b2c510b1f0f1364c)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:25:24.146566 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fbfb72b-cb91-41a2-9710-e9939bd6e2e1 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:25:24.156099 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-040c71ab-86bd-42eb-bc47-1f75e09b527b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'be01f210b2d643cb9678f4cf1802c23e'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:25:24.156791 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-040c71ab-86bd-42eb-bc47-1f75e09b527b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'be01f210b2d643cb9678f4cf1802c23e'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:25:24.161106 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1038/2098] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 15 12:25:24 2022] PUT /identity/v3/projects/f23572583527499b9aecb3ee291c60cc/users/bcf49aeb19034e4daf5c5eea501eb727/roles/9c769b5c82884766b2c510b1f0f1364c => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:25:24.164502 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-040c71ab-86bd-42eb-bc47-1f75e09b527b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'be01f210b2d643cb9678f4cf1802c23e'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:25:24.165036 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fbfb72b-cb91-41a2-9710-e9939bd6e2e1 admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:25:24.167348 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-7fbfb72b-cb91-41a2-9710-e9939bd6e2e1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:25:24.171148 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1061/2099] 10.222.0.15 () {58 vars in 1224 bytes} [Thu Sep 15 12:25:24 2022] DELETE /identity/v3/users/be01f210b2d643cb9678f4cf1802c23e => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:25:24.175441 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5db5eac-0fd6-40d3-a7f9-c3ecaca729fc None None] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:25:24.175756 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5db5eac-0fd6-40d3-a7f9-c3ecaca729fc None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:25:24.176034 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5db5eac-0fd6-40d3-a7f9-c3ecaca729fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:25:24.195660 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.auth.core [None req-f5db5eac-0fd6-40d3-a7f9-c3ecaca729fc None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81449) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:25:24.196896 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-f5db5eac-0fd6-40d3-a7f9-c3ecaca729fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:25:24.212508 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:25:24.233171 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fbfb72b-cb91-41a2-9710-e9939bd6e2e1 admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:25:24.258633 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1062/2100] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 12:25:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 12:25:24.268397 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5db5eac-0fd6-40d3-a7f9-c3ecaca729fc None None] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:25:24.269769 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-f5db5eac-0fd6-40d3-a7f9-c3ecaca729fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:25:24.290939 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9881697-1d59-4f9d-b4fc-1faaa1d09fc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:25:24.292116 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9881697-1d59-4f9d-b4fc-1faaa1d09fc8 admin admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:25:24.292393 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9881697-1d59-4f9d-b4fc-1faaa1d09fc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:25:24.292662 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9881697-1d59-4f9d-b4fc-1faaa1d09fc8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:25:24.293327 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9881697-1d59-4f9d-b4fc-1faaa1d09fc8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:25:24.294975 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9881697-1d59-4f9d-b4fc-1faaa1d09fc8 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:25:24.305218 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1039/2101] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 12:25:24 2022] GET /identity/v3/roles => generated 1420 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 15 12:25:24.309218 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9881697-1d59-4f9d-b4fc-1faaa1d09fc8 admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:25:24.318682 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9881697-1d59-4f9d-b4fc-1faaa1d09fc8 admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:25:24.323309 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9485fb-cbbd-4037-8278-342e0df3e6c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:25:24.323978 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9485fb-cbbd-4037-8278-342e0df3e6c6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:25:24.324262 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9485fb-cbbd-4037-8278-342e0df3e6c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:25:24.324580 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9485fb-cbbd-4037-8278-342e0df3e6c6 admin admin] PATH_INFO: `/v3/projects/f23572583527499b9aecb3ee291c60cc/users/bcf49aeb19034e4daf5c5eea501eb727/roles/9c769b5c82884766b2c510b1f0f1364c` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:25:24.326534 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9485fb-cbbd-4037-8278-342e0df3e6c6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f23572583527499b9aecb3ee291c60cc, user_id=bcf49aeb19034e4daf5c5eea501eb727, role_id=9c769b5c82884766b2c510b1f0f1364c)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:25:24.328054 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5db5eac-0fd6-40d3-a7f9-c3ecaca729fc None None] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:25:24.329184 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9485fb-cbbd-4037-8278-342e0df3e6c6 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:25:24.331725 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b472f2-5841-44aa-8ba5-87dd85053eb7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfbb4d6809064f9cb9e06626ed7d6814', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '540a32ef88764e8c8d9201ae22f6ddaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:25:24.332388 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b472f2-5841-44aa-8ba5-87dd85053eb7 service neutron] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:25:24.332671 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b472f2-5841-44aa-8ba5-87dd85053eb7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:25:24.333052 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b472f2-5841-44aa-8ba5-87dd85053eb7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:25:24.340305 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1040/2102] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 15 12:25:24 2022] PUT /identity/v3/projects/f23572583527499b9aecb3ee291c60cc/users/bcf49aeb19034e4daf5c5eea501eb727/roles/9c769b5c82884766b2c510b1f0f1364c => generated 0 bytes in 31 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 12:25:24.346988 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b472f2-5841-44aa-8ba5-87dd85053eb7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:25:24.348437 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b472f2-5841-44aa-8ba5-87dd85053eb7 service neutron] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:25:24.351594 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a235d88a-822c-4bd3-8d89-bd38b9981548 None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:25:24.351886 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a235d88a-822c-4bd3-8d89-bd38b9981548 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:25:24.352161 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a235d88a-822c-4bd3-8d89-bd38b9981548 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:25:24.354727 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:25:24.360745 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1063/2103] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 12:25:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 15 12:25:24.383611 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-a235d88a-822c-4bd3-8d89-bd38b9981548 None None] MFA Rules not processed for user `bcf49aeb19034e4daf5c5eea501eb727`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:25:24.384962 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-a235d88a-822c-4bd3-8d89-bd38b9981548 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:25:24.454740 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1041/2104] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 12:25:24 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 12:25:24.467313 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b472f2-5841-44aa-8ba5-87dd85053eb7 service neutron] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:25:24.469431 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-31b472f2-5841-44aa-8ba5-87dd85053eb7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:25:24.537334 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b472f2-5841-44aa-8ba5-87dd85053eb7 service neutron] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:25:24.588925 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2904c5d5-0428-49ab-a3db-ee14b8900fdb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8191d3ccb874529b32f8d0026db629d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '540a32ef88764e8c8d9201ae22f6ddaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:25:24.589668 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2904c5d5-0428-49ab-a3db-ee14b8900fdb service cinder] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:25:24.590043 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2904c5d5-0428-49ab-a3db-ee14b8900fdb service cinder] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:25:24.590677 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2904c5d5-0428-49ab-a3db-ee14b8900fdb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:25:24.600544 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2904c5d5-0428-49ab-a3db-ee14b8900fdb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:25:24.602233 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2904c5d5-0428-49ab-a3db-ee14b8900fdb service cinder] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:25:24.614365 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1064/2105] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 12:25:24 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 12:25:24.654458 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a235d88a-822c-4bd3-8d89-bd38b9981548 None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:25:24.656246 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-a235d88a-822c-4bd3-8d89-bd38b9981548 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:25:24.727950 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:25:24.732436 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a235d88a-822c-4bd3-8d89-bd38b9981548 None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:25:24.742896 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d093fdc5-40fe-42fd-880d-3ef7500f8dc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:25:24.743537 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d093fdc5-40fe-42fd-880d-3ef7500f8dc3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:25:24.743833 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d093fdc5-40fe-42fd-880d-3ef7500f8dc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:25:24.744097 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d093fdc5-40fe-42fd-880d-3ef7500f8dc3 admin admin] PATH_INFO: `/v3/projects/762613f23f714b489992a2dd539058d4` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:25:24.745285 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d093fdc5-40fe-42fd-880d-3ef7500f8dc3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=762613f23f714b489992a2dd539058d4)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:25:24.747194 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d093fdc5-40fe-42fd-880d-3ef7500f8dc3 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:25:24.799090 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-d093fdc5-40fe-42fd-880d-3ef7500f8dc3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '762613f23f714b489992a2dd539058d4', 'request_id': 'req-d093fdc5-40fe-42fd-880d-3ef7500f8dc3'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:25:24.815819 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-d093fdc5-40fe-42fd-880d-3ef7500f8dc3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '762613f23f714b489992a2dd539058d4', 'request_id': 'req-d093fdc5-40fe-42fd-880d-3ef7500f8dc3'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:25:24.828949 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-d093fdc5-40fe-42fd-880d-3ef7500f8dc3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '762613f23f714b489992a2dd539058d4', 'request_id': 'req-d093fdc5-40fe-42fd-880d-3ef7500f8dc3'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:25:24.845600 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-d093fdc5-40fe-42fd-880d-3ef7500f8dc3 admin admin] The token cache is being invalidate because project 762613f23f714b489992a2dd539058d4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81448) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 12:25:24.845935 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-d093fdc5-40fe-42fd-880d-3ef7500f8dc3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:25:24.846567 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-d093fdc5-40fe-42fd-880d-3ef7500f8dc3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:25:24.852585 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1042/2106] 10.222.0.15 () {58 vars in 1233 bytes} [Thu Sep 15 12:25:24 2022] DELETE /identity/v3/projects/762613f23f714b489992a2dd539058d4 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:25:24.852695 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2904c5d5-0428-49ab-a3db-ee14b8900fdb service cinder] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:25:24.864438 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2904c5d5-0428-49ab-a3db-ee14b8900fdb service cinder] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:25:24.928383 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3ef6c2-ddf6-4307-80a3-c5e489faadaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:25:24.929070 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3ef6c2-ddf6-4307-80a3-c5e489faadaa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:25:24.929352 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3ef6c2-ddf6-4307-80a3-c5e489faadaa admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:25:24.929623 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3ef6c2-ddf6-4307-80a3-c5e489faadaa admin admin] PATH_INFO: `/v3/users/034c9609640740659ca1e86fd13f0044` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:25:24.930763 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3ef6c2-ddf6-4307-80a3-c5e489faadaa admin admin] RBAC: Authorizing `identity:delete_user(user_id=034c9609640740659ca1e86fd13f0044)` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:25:24.932742 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3ef6c2-ddf6-4307-80a3-c5e489faadaa admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:25:24.991957 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-1b3ef6c2-ddf6-4307-80a3-c5e489faadaa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '034c9609640740659ca1e86fd13f0044'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:25:25.000837 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-1b3ef6c2-ddf6-4307-80a3-c5e489faadaa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '034c9609640740659ca1e86fd13f0044'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:25:25.006436 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:25:25.016301 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-1b3ef6c2-ddf6-4307-80a3-c5e489faadaa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '034c9609640740659ca1e86fd13f0044'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:25:25.017241 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-1b3ef6c2-ddf6-4307-80a3-c5e489faadaa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '034c9609640740659ca1e86fd13f0044'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:25:25.024333 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-1b3ef6c2-ddf6-4307-80a3-c5e489faadaa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '034c9609640740659ca1e86fd13f0044'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:25:25.030501 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1065/2107] 10.222.0.15 () {58 vars in 1224 bytes} [Thu Sep 15 12:25:24 2022] DELETE /identity/v3/users/034c9609640740659ca1e86fd13f0044 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:25:25.067705 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d093fdc5-40fe-42fd-880d-3ef7500f8dc3 admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:25:25.069252 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-d093fdc5-40fe-42fd-880d-3ef7500f8dc3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:25:25.083246 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d093fdc5-40fe-42fd-880d-3ef7500f8dc3 admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:25:25.147238 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef518e6-65a2-42ff-8fd5-00b807d211aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:25:25.147889 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef518e6-65a2-42ff-8fd5-00b807d211aa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:25:25.148518 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef518e6-65a2-42ff-8fd5-00b807d211aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:25:25.148518 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef518e6-65a2-42ff-8fd5-00b807d211aa admin admin] PATH_INFO: `/v3/users/de293cfdebde41b089afb0cb3ed061e7` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:25:25.150083 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef518e6-65a2-42ff-8fd5-00b807d211aa admin admin] RBAC: Authorizing `identity:delete_user(user_id=de293cfdebde41b089afb0cb3ed061e7)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:25:25.152605 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef518e6-65a2-42ff-8fd5-00b807d211aa admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:25:25.217843 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-eef518e6-65a2-42ff-8fd5-00b807d211aa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'de293cfdebde41b089afb0cb3ed061e7'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:25:25.230536 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-eef518e6-65a2-42ff-8fd5-00b807d211aa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'de293cfdebde41b089afb0cb3ed061e7'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:25:25.239207 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-eef518e6-65a2-42ff-8fd5-00b807d211aa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'de293cfdebde41b089afb0cb3ed061e7'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:25:25.239827 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-eef518e6-65a2-42ff-8fd5-00b807d211aa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'de293cfdebde41b089afb0cb3ed061e7'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:25:25.247481 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-eef518e6-65a2-42ff-8fd5-00b807d211aa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'de293cfdebde41b089afb0cb3ed061e7'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:25:25.253286 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1043/2108] 10.222.0.15 () {58 vars in 1224 bytes} [Thu Sep 15 12:25:25 2022] DELETE /identity/v3/users/de293cfdebde41b089afb0cb3ed061e7 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:25:25.257770 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e2dd16-284d-430a-870d-2efd9e6e30a7 None None] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:25:25.258087 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e2dd16-284d-430a-870d-2efd9e6e30a7 None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:25:25.258364 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e2dd16-284d-430a-870d-2efd9e6e30a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:25:25.278125 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.auth.core [None req-a2e2dd16-284d-430a-870d-2efd9e6e30a7 None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81449) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:25:25.279327 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-a2e2dd16-284d-430a-870d-2efd9e6e30a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:25:25.325839 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef518e6-65a2-42ff-8fd5-00b807d211aa admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:25:25.327310 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-eef518e6-65a2-42ff-8fd5-00b807d211aa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:25:25.339405 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:25:25.340665 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1066/2109] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 12:25:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 12:25:25.354151 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e2dd16-284d-430a-870d-2efd9e6e30a7 None None] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:25:25.355782 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-a2e2dd16-284d-430a-870d-2efd9e6e30a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:25:25.409667 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef518e6-65a2-42ff-8fd5-00b807d211aa admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:25:25.415009 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0b8393-39e8-49cf-a1a9-9e784ea87e8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:25:25.416676 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0b8393-39e8-49cf-a1a9-9e784ea87e8e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:25:25.416994 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0b8393-39e8-49cf-a1a9-9e784ea87e8e admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:25:25.417273 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0b8393-39e8-49cf-a1a9-9e784ea87e8e admin admin] PATH_INFO: `/v3/projects/dcd1fdb7a9824f59a5118e20466066d2` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:25:25.420243 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0b8393-39e8-49cf-a1a9-9e784ea87e8e admin admin] RBAC: Authorizing `identity:delete_project(project_id=dcd1fdb7a9824f59a5118e20466066d2)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:25:25.423092 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0b8393-39e8-49cf-a1a9-9e784ea87e8e admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:25:25.428848 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e2dd16-284d-430a-870d-2efd9e6e30a7 None None] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:25:25.434315 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125e591d-5426-4cc8-b708-c1fe4e881a05 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfbb4d6809064f9cb9e06626ed7d6814', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '540a32ef88764e8c8d9201ae22f6ddaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:25:25.435322 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125e591d-5426-4cc8-b708-c1fe4e881a05 service neutron] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:25:25.435616 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125e591d-5426-4cc8-b708-c1fe4e881a05 service neutron] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:25:25.436526 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125e591d-5426-4cc8-b708-c1fe4e881a05 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:25:25.448046 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125e591d-5426-4cc8-b708-c1fe4e881a05 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:25:25.449518 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125e591d-5426-4cc8-b708-c1fe4e881a05 service neutron] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:25:25.460333 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1067/2110] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 12:25:25 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 12:25:25.478455 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-4e0b8393-39e8-49cf-a1a9-9e784ea87e8e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dcd1fdb7a9824f59a5118e20466066d2', 'request_id': 'req-4e0b8393-39e8-49cf-a1a9-9e784ea87e8e'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:25:25.487019 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-4e0b8393-39e8-49cf-a1a9-9e784ea87e8e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dcd1fdb7a9824f59a5118e20466066d2', 'request_id': 'req-4e0b8393-39e8-49cf-a1a9-9e784ea87e8e'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:25:25.493984 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-4e0b8393-39e8-49cf-a1a9-9e784ea87e8e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dcd1fdb7a9824f59a5118e20466066d2', 'request_id': 'req-4e0b8393-39e8-49cf-a1a9-9e784ea87e8e'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:25:25.506495 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-4e0b8393-39e8-49cf-a1a9-9e784ea87e8e admin admin] The token cache is being invalidate because project dcd1fdb7a9824f59a5118e20466066d2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81448) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 12:25:25.506855 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-4e0b8393-39e8-49cf-a1a9-9e784ea87e8e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:25:25.507534 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-4e0b8393-39e8-49cf-a1a9-9e784ea87e8e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:25:25.509212 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1044/2111] 10.222.0.15 () {58 vars in 1233 bytes} [Thu Sep 15 12:25:25 2022] DELETE /identity/v3/projects/dcd1fdb7a9824f59a5118e20466066d2 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:25:25.513446 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125e591d-5426-4cc8-b708-c1fe4e881a05 service neutron] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:25:25.523653 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125e591d-5426-4cc8-b708-c1fe4e881a05 service neutron] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:25:25.583591 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea89221-dbcd-469f-8bec-c5ee3d70e40b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:25:25.584228 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea89221-dbcd-469f-8bec-c5ee3d70e40b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:25:25.584508 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea89221-dbcd-469f-8bec-c5ee3d70e40b admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:25:25.584798 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea89221-dbcd-469f-8bec-c5ee3d70e40b admin admin] PATH_INFO: `/v3/users/01f36333b8fa4550b42e588eabffb783` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:25:25.585930 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea89221-dbcd-469f-8bec-c5ee3d70e40b admin admin] RBAC: Authorizing `identity:delete_user(user_id=01f36333b8fa4550b42e588eabffb783)` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:25:25.587841 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea89221-dbcd-469f-8bec-c5ee3d70e40b admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:25:25.612736 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:25:25.648274 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-fea89221-dbcd-469f-8bec-c5ee3d70e40b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '01f36333b8fa4550b42e588eabffb783'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:25:25.675483 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-fea89221-dbcd-469f-8bec-c5ee3d70e40b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '01f36333b8fa4550b42e588eabffb783'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:25:25.689109 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-fea89221-dbcd-469f-8bec-c5ee3d70e40b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '01f36333b8fa4550b42e588eabffb783'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:25:25.689788 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-fea89221-dbcd-469f-8bec-c5ee3d70e40b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '01f36333b8fa4550b42e588eabffb783'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:25:25.696876 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-fea89221-dbcd-469f-8bec-c5ee3d70e40b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '01f36333b8fa4550b42e588eabffb783'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:25:25.703444 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1068/2112] 10.222.0.15 () {58 vars in 1224 bytes} [Thu Sep 15 12:25:25 2022] DELETE /identity/v3/users/01f36333b8fa4550b42e588eabffb783 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:25:25.757487 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0b8393-39e8-49cf-a1a9-9e784ea87e8e admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:25:25.759038 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-4e0b8393-39e8-49cf-a1a9-9e784ea87e8e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:25:25.773474 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0b8393-39e8-49cf-a1a9-9e784ea87e8e admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:25:25.840664 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6709af7-fd02-4d21-9dde-196aa32addcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:25:25.841334 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6709af7-fd02-4d21-9dde-196aa32addcc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:25:25.841616 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6709af7-fd02-4d21-9dde-196aa32addcc admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:25:25.841887 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6709af7-fd02-4d21-9dde-196aa32addcc admin admin] PATH_INFO: `/v3/projects/3d4e37afdac74a26a14355663bad78ea` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:25:25.843011 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6709af7-fd02-4d21-9dde-196aa32addcc admin admin] RBAC: Authorizing `identity:delete_project(project_id=3d4e37afdac74a26a14355663bad78ea)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:25:25.845184 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6709af7-fd02-4d21-9dde-196aa32addcc admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:25:25.890170 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:25:25.898488 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-f6709af7-fd02-4d21-9dde-196aa32addcc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3d4e37afdac74a26a14355663bad78ea', 'request_id': 'req-f6709af7-fd02-4d21-9dde-196aa32addcc'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:25:25.907733 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-f6709af7-fd02-4d21-9dde-196aa32addcc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3d4e37afdac74a26a14355663bad78ea', 'request_id': 'req-f6709af7-fd02-4d21-9dde-196aa32addcc'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:25:25.913598 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-f6709af7-fd02-4d21-9dde-196aa32addcc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3d4e37afdac74a26a14355663bad78ea', 'request_id': 'req-f6709af7-fd02-4d21-9dde-196aa32addcc'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:25:25.925208 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-f6709af7-fd02-4d21-9dde-196aa32addcc admin admin] The token cache is being invalidate because project 3d4e37afdac74a26a14355663bad78ea was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81448) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 12:25:25.925491 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-f6709af7-fd02-4d21-9dde-196aa32addcc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:25:25.926136 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-f6709af7-fd02-4d21-9dde-196aa32addcc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:25:25.927841 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1045/2113] 10.222.0.15 () {58 vars in 1233 bytes} [Thu Sep 15 12:25:25 2022] DELETE /identity/v3/projects/3d4e37afdac74a26a14355663bad78ea => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:25:25.932554 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea89221-dbcd-469f-8bec-c5ee3d70e40b admin admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:25:25.934184 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-fea89221-dbcd-469f-8bec-c5ee3d70e40b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:25:25.946919 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea89221-dbcd-469f-8bec-c5ee3d70e40b admin admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:25:26.016189 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31ee48f-0666-41c4-94be-43e4829bf1f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:25:26.016964 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31ee48f-0666-41c4-94be-43e4829bf1f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:25:26.017361 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31ee48f-0666-41c4-94be-43e4829bf1f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:25:26.017543 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31ee48f-0666-41c4-94be-43e4829bf1f6 admin admin] PATH_INFO: `/v3/users/bcf49aeb19034e4daf5c5eea501eb727` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:25:26.018673 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31ee48f-0666-41c4-94be-43e4829bf1f6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bcf49aeb19034e4daf5c5eea501eb727)` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:25:26.020696 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31ee48f-0666-41c4-94be-43e4829bf1f6 admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:25:26.028632 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6709af7-fd02-4d21-9dde-196aa32addcc admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:25:26.030412 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-f6709af7-fd02-4d21-9dde-196aa32addcc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:25:26.043554 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6709af7-fd02-4d21-9dde-196aa32addcc admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:25:26.049750 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ebb618-019c-4d1c-89b4-f00a4bbf4231 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:25:26.050517 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ebb618-019c-4d1c-89b4-f00a4bbf4231 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:25:26.050982 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ebb618-019c-4d1c-89b4-f00a4bbf4231 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:25:26.051456 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ebb618-019c-4d1c-89b4-f00a4bbf4231 admin admin] PATH_INFO: `/v3/projects/fad3b3ceac6f454d9e05dec496e58eec` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:25:26.053151 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ebb618-019c-4d1c-89b4-f00a4bbf4231 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fad3b3ceac6f454d9e05dec496e58eec)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:25:26.055432 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ebb618-019c-4d1c-89b4-f00a4bbf4231 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:25:26.073751 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-c31ee48f-0666-41c4-94be-43e4829bf1f6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bcf49aeb19034e4daf5c5eea501eb727'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:25:26.082119 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-c31ee48f-0666-41c4-94be-43e4829bf1f6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bcf49aeb19034e4daf5c5eea501eb727'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:25:26.096869 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-c31ee48f-0666-41c4-94be-43e4829bf1f6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bcf49aeb19034e4daf5c5eea501eb727'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:25:26.097584 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-c31ee48f-0666-41c4-94be-43e4829bf1f6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bcf49aeb19034e4daf5c5eea501eb727'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:25:26.105443 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-c31ee48f-0666-41c4-94be-43e4829bf1f6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bcf49aeb19034e4daf5c5eea501eb727'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:25:26.109520 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-89ebb618-019c-4d1c-89b4-f00a4bbf4231 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fad3b3ceac6f454d9e05dec496e58eec', 'request_id': 'req-89ebb618-019c-4d1c-89b4-f00a4bbf4231'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:25:26.112810 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1069/2114] 10.222.0.15 () {58 vars in 1224 bytes} [Thu Sep 15 12:25:25 2022] DELETE /identity/v3/users/bcf49aeb19034e4daf5c5eea501eb727 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 12:25:26.118012 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-89ebb618-019c-4d1c-89b4-f00a4bbf4231 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fad3b3ceac6f454d9e05dec496e58eec', 'request_id': 'req-89ebb618-019c-4d1c-89b4-f00a4bbf4231'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:25:26.124655 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-89ebb618-019c-4d1c-89b4-f00a4bbf4231 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fad3b3ceac6f454d9e05dec496e58eec', 'request_id': 'req-89ebb618-019c-4d1c-89b4-f00a4bbf4231'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:25:26.136699 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-89ebb618-019c-4d1c-89b4-f00a4bbf4231 admin admin] The token cache is being invalidate because project fad3b3ceac6f454d9e05dec496e58eec was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81448) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 12:25:26.137028 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-89ebb618-019c-4d1c-89b4-f00a4bbf4231 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:25:26.137699 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-89ebb618-019c-4d1c-89b4-f00a4bbf4231 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:25:26.139300 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1046/2115] 10.222.0.15 () {58 vars in 1233 bytes} [Thu Sep 15 12:25:26 2022] DELETE /identity/v3/projects/fad3b3ceac6f454d9e05dec496e58eec => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 12:25:26.144192 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623c8f73-90bf-481c-bec5-bd5cd1f3e909 None None] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:25:26.144421 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623c8f73-90bf-481c-bec5-bd5cd1f3e909 None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:25:26.144690 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623c8f73-90bf-481c-bec5-bd5cd1f3e909 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:25:26.167407 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.auth.core [None req-623c8f73-90bf-481c-bec5-bd5cd1f3e909 None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81449) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:25:26.168748 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-623c8f73-90bf-481c-bec5-bd5cd1f3e909 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:25:26.182389 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:25:26.240829 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1070/2116] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 12:25:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:25:26.249343 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c094175f-decd-4ebc-a109-80bf58c4d1c3 None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:25:26.249626 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c094175f-decd-4ebc-a109-80bf58c4d1c3 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:25:26.249895 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c094175f-decd-4ebc-a109-80bf58c4d1c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:25:26.273104 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-c094175f-decd-4ebc-a109-80bf58c4d1c3 None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:25:26.274463 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-c094175f-decd-4ebc-a109-80bf58c4d1c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:25:26.283276 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1047/2117] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 12:25:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:25:26.287507 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623c8f73-90bf-481c-bec5-bd5cd1f3e909 None None] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:25:26.298065 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623c8f73-90bf-481c-bec5-bd5cd1f3e909 None None] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:25:26.315682 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec12f899-0fe1-49af-b80e-4a6c0b2d7d91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:25:26.316424 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec12f899-0fe1-49af-b80e-4a6c0b2d7d91 admin admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:25:26.316934 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec12f899-0fe1-49af-b80e-4a6c0b2d7d91 admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:25:26.317294 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec12f899-0fe1-49af-b80e-4a6c0b2d7d91 admin admin] PATH_INFO: `/v3/domains` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:25:26.318110 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec12f899-0fe1-49af-b80e-4a6c0b2d7d91 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81449) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:25:26.318396 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec12f899-0fe1-49af-b80e-4a6c0b2d7d91 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:25:26.320835 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec12f899-0fe1-49af-b80e-4a6c0b2d7d91 admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:25:26.336830 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1071/2118] 10.222.0.15 () {58 vars in 1153 bytes} [Thu Sep 15 12:25:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:25:26.338122 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c094175f-decd-4ebc-a109-80bf58c4d1c3 None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:25:26.347920 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c094175f-decd-4ebc-a109-80bf58c4d1c3 None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:25:26.352657 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a518143-32f3-4436-b9c0-f9dfc904739b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:25:26.353395 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a518143-32f3-4436-b9c0-f9dfc904739b admin admin] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:25:26.354266 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a518143-32f3-4436-b9c0-f9dfc904739b admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:25:26.354599 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a518143-32f3-4436-b9c0-f9dfc904739b admin admin] PATH_INFO: `/v3/projects` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:25:26.355305 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a518143-32f3-4436-b9c0-f9dfc904739b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:25:26.357275 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a518143-32f3-4436-b9c0-f9dfc904739b admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:25:26.385918 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1048/2119] 10.222.0.15 () {60 vars in 1153 bytes} [Thu Sep 15 12:25:26 2022] POST /identity/v3/projects => generated 371 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 12:25:26.390447 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec12f899-0fe1-49af-b80e-4a6c0b2d7d91 admin admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:25:26.396078 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a518143-32f3-4436-b9c0-f9dfc904739b admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:25:26.397721 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-4a518143-32f3-4436-b9c0-f9dfc904739b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:25:26.400881 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec12f899-0fe1-49af-b80e-4a6c0b2d7d91 admin admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:25:26.468217 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9155f46e-1d62-4e4d-b94a-21311ab8c286 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:25:26.468864 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9155f46e-1d62-4e4d-b94a-21311ab8c286 admin admin] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:25:26.469176 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9155f46e-1d62-4e4d-b94a-21311ab8c286 admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:25:26.469448 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9155f46e-1d62-4e4d-b94a-21311ab8c286 admin admin] PATH_INFO: `/v3/users` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:25:26.470371 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9155f46e-1d62-4e4d-b94a-21311ab8c286 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:25:26.472274 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9155f46e-1d62-4e4d-b94a-21311ab8c286 admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:25:26.474916 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a518143-32f3-4436-b9c0-f9dfc904739b admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:25:26.480232 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6cac889-30c0-492c-b3df-7b42d8a39957 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:25:26.481007 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cac889-30c0-492c-b3df-7b42d8a39957 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:25:26.481182 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cac889-30c0-492c-b3df-7b42d8a39957 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:25:26.481441 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cac889-30c0-492c-b3df-7b42d8a39957 admin admin] PATH_INFO: `/v3/projects/f23572583527499b9aecb3ee291c60cc` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:25:26.482550 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6cac889-30c0-492c-b3df-7b42d8a39957 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f23572583527499b9aecb3ee291c60cc)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:25:26.484634 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6cac889-30c0-492c-b3df-7b42d8a39957 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:25:26.493634 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1072/2120] 10.222.0.15 () {60 vars in 1144 bytes} [Thu Sep 15 12:25:26 2022] POST /identity/v3/users => generated 286 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 12:25:26.499181 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9155f46e-1d62-4e4d-b94a-21311ab8c286 admin admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:25:26.508513 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9155f46e-1d62-4e4d-b94a-21311ab8c286 admin admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:25:26.513005 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:25:26.513211 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d217c6-c01e-4051-8d91-0c0368cc074b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:25:26.513794 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d217c6-c01e-4051-8d91-0c0368cc074b admin admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:25:26.514070 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d217c6-c01e-4051-8d91-0c0368cc074b admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:25:26.514334 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d217c6-c01e-4051-8d91-0c0368cc074b admin admin] PATH_INFO: `/v3/roles` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:25:26.515135 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d217c6-c01e-4051-8d91-0c0368cc074b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:25:26.517179 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d217c6-c01e-4051-8d91-0c0368cc074b admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:25:26.529029 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1073/2121] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 12:25:26 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 15 12:25:26.533257 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d217c6-c01e-4051-8d91-0c0368cc074b admin admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:25:26.538997 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-d6cac889-30c0-492c-b3df-7b42d8a39957 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f23572583527499b9aecb3ee291c60cc', 'request_id': 'req-d6cac889-30c0-492c-b3df-7b42d8a39957'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:25:26.543776 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d217c6-c01e-4051-8d91-0c0368cc074b admin admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:25:26.548243 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-d6cac889-30c0-492c-b3df-7b42d8a39957 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f23572583527499b9aecb3ee291c60cc', 'request_id': 'req-d6cac889-30c0-492c-b3df-7b42d8a39957'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:25:26.554177 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-d6cac889-30c0-492c-b3df-7b42d8a39957 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f23572583527499b9aecb3ee291c60cc', 'request_id': 'req-d6cac889-30c0-492c-b3df-7b42d8a39957'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:25:26.565818 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-d6cac889-30c0-492c-b3df-7b42d8a39957 admin admin] The token cache is being invalidate because project f23572583527499b9aecb3ee291c60cc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81448) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 12:25:26.566108 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-d6cac889-30c0-492c-b3df-7b42d8a39957 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:25:26.566854 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-d6cac889-30c0-492c-b3df-7b42d8a39957 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:25:26.568362 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1049/2122] 10.222.0.15 () {58 vars in 1233 bytes} [Thu Sep 15 12:25:26 2022] DELETE /identity/v3/projects/f23572583527499b9aecb3ee291c60cc => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 12:25:26.573687 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0b573f-516a-4d95-a23e-d3f022a1856f None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:25:26.573972 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0b573f-516a-4d95-a23e-d3f022a1856f None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:25:26.574436 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0b573f-516a-4d95-a23e-d3f022a1856f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:25:26.594405 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-1a0b573f-516a-4d95-a23e-d3f022a1856f None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:25:26.595881 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-1a0b573f-516a-4d95-a23e-d3f022a1856f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:25:26.604849 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd638d5-dc81-4d11-857b-663a461d3d25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:25:26.605465 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd638d5-dc81-4d11-857b-663a461d3d25 admin admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:25:26.605746 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd638d5-dc81-4d11-857b-663a461d3d25 admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:25:26.606015 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd638d5-dc81-4d11-857b-663a461d3d25 admin admin] PATH_INFO: `/v3/roles` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:25:26.606656 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd638d5-dc81-4d11-857b-663a461d3d25 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:25:26.608264 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd638d5-dc81-4d11-857b-663a461d3d25 admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:25:26.620828 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1074/2123] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 12:25:26 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 12:25:26.622578 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd638d5-dc81-4d11-857b-663a461d3d25 admin admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:25:26.624224 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-abd638d5-dc81-4d11-857b-663a461d3d25 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:25:26.637759 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd638d5-dc81-4d11-857b-663a461d3d25 admin admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:25:26.642922 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4b4cd2-5aaa-4a61-a2c1-4fd0721fa632 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:25:26.643578 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4b4cd2-5aaa-4a61-a2c1-4fd0721fa632 admin admin] REQUEST_METHOD: `PUT` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:25:26.643863 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4b4cd2-5aaa-4a61-a2c1-4fd0721fa632 admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:25:26.644159 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4b4cd2-5aaa-4a61-a2c1-4fd0721fa632 admin admin] PATH_INFO: `/v3/projects/1833fa8600d84bcb892dfcab47e03c17/users/9699fd2d69f1499ea38a861085bd1821/roles/0679d6628a25438d90dc4bdc0401eda2` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:25:26.660108 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4b4cd2-5aaa-4a61-a2c1-4fd0721fa632 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1833fa8600d84bcb892dfcab47e03c17, user_id=9699fd2d69f1499ea38a861085bd1821, role_id=0679d6628a25438d90dc4bdc0401eda2)` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:25:26.662744 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4b4cd2-5aaa-4a61-a2c1-4fd0721fa632 admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:25:26.664921 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1050/2124] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 12:25:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 12:25:26.674411 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7306bbe1-fd34-408f-b624-d9e12c1a3941 None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:25:26.674746 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7306bbe1-fd34-408f-b624-d9e12c1a3941 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:25:26.675170 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7306bbe1-fd34-408f-b624-d9e12c1a3941 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:25:26.677201 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:25:26.683537 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1075/2125] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 15 12:25:26 2022] PUT /identity/v3/projects/1833fa8600d84bcb892dfcab47e03c17/users/9699fd2d69f1499ea38a861085bd1821/roles/0679d6628a25438d90dc4bdc0401eda2 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 12:25:26.695858 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175303a3-a806-4734-956e-a39137a96c4b None None] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:25:26.696147 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175303a3-a806-4734-956e-a39137a96c4b None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:25:26.696734 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175303a3-a806-4734-956e-a39137a96c4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:25:26.703174 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-7306bbe1-fd34-408f-b624-d9e12c1a3941 None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:25:26.704621 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-7306bbe1-fd34-408f-b624-d9e12c1a3941 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:25:26.730463 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.auth.core [None req-175303a3-a806-4734-956e-a39137a96c4b None None] MFA Rules not processed for user `9699fd2d69f1499ea38a861085bd1821`. Rule list: `[]` (Enabled: `True`). {{(pid=81449) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:25:26.732463 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-175303a3-a806-4734-956e-a39137a96c4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:25:26.788613 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1051/2126] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 12:25:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:25:26.793420 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7306bbe1-fd34-408f-b624-d9e12c1a3941 None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:25:26.803212 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7306bbe1-fd34-408f-b624-d9e12c1a3941 None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:25:26.808048 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6104de5-ff1f-486f-ac5b-b1278a4c8ebe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:25:26.808648 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6104de5-ff1f-486f-ac5b-b1278a4c8ebe admin admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:25:26.808957 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6104de5-ff1f-486f-ac5b-b1278a4c8ebe admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:25:26.809230 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6104de5-ff1f-486f-ac5b-b1278a4c8ebe admin admin] PATH_INFO: `/v3/domains` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:25:26.809877 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6104de5-ff1f-486f-ac5b-b1278a4c8ebe admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:25:26.810178 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6104de5-ff1f-486f-ac5b-b1278a4c8ebe admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:25:26.811806 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6104de5-ff1f-486f-ac5b-b1278a4c8ebe admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:25:26.812463 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1076/2127] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 12:25:26 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:25:26.827229 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1052/2128] 10.222.0.15 () {58 vars in 1153 bytes} [Thu Sep 15 12:25:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 12:25:26.827399 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175303a3-a806-4734-956e-a39137a96c4b None None] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:25:26.827399 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-175303a3-a806-4734-956e-a39137a96c4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:25:26.828183 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6104de5-ff1f-486f-ac5b-b1278a4c8ebe admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:25:26.840785 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6104de5-ff1f-486f-ac5b-b1278a4c8ebe admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:25:26.843952 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6cef00f-f5ee-40d8-84dc-3ff9baa0f571 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:25:26.844564 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cef00f-f5ee-40d8-84dc-3ff9baa0f571 admin admin] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:25:26.845765 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cef00f-f5ee-40d8-84dc-3ff9baa0f571 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:25:26.845765 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6cef00f-f5ee-40d8-84dc-3ff9baa0f571 admin admin] PATH_INFO: `/v3/projects` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:25:26.846031 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6cef00f-f5ee-40d8-84dc-3ff9baa0f571 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:25:26.847939 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6cef00f-f5ee-40d8-84dc-3ff9baa0f571 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:25:26.877102 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1053/2129] 10.222.0.15 () {60 vars in 1153 bytes} [Thu Sep 15 12:25:26 2022] POST /identity/v3/projects => generated 387 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 15 12:25:26.881291 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6cef00f-f5ee-40d8-84dc-3ff9baa0f571 admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:25:26.895114 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175303a3-a806-4734-956e-a39137a96c4b None None] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:25:26.905628 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=92,keystone:DELETE=6,keystone:INSERT=2 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:25:26.948770 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778148f2-6115-4d95-b10c-4d924a588e3e service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3aed8f78903f44cda9e5254aa46cf2b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '540a32ef88764e8c8d9201ae22f6ddaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:25:26.949418 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778148f2-6115-4d95-b10c-4d924a588e3e service glance] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:25:26.949698 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778148f2-6115-4d95-b10c-4d924a588e3e service glance] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:25:26.949970 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778148f2-6115-4d95-b10c-4d924a588e3e service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:25:26.951422 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6cef00f-f5ee-40d8-84dc-3ff9baa0f571 admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:25:26.962251 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d2586d-0c4c-42bf-9d8b-c685c65a3abe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:25:26.963859 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d2586d-0c4c-42bf-9d8b-c685c65a3abe admin admin] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:25:26.964141 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d2586d-0c4c-42bf-9d8b-c685c65a3abe admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:25:26.964432 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d2586d-0c4c-42bf-9d8b-c685c65a3abe admin admin] PATH_INFO: `/v3/users` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:25:26.965124 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d2586d-0c4c-42bf-9d8b-c685c65a3abe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:25:26.967023 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d2586d-0c4c-42bf-9d8b-c685c65a3abe admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:25:26.990877 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1054/2130] 10.222.0.15 () {60 vars in 1144 bytes} [Thu Sep 15 12:25:26 2022] POST /identity/v3/users => generated 294 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 12:25:26.994794 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d2586d-0c4c-42bf-9d8b-c685c65a3abe admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:25:27.004522 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d2586d-0c4c-42bf-9d8b-c685c65a3abe admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:25:27.009684 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d158fa1f-5622-4e06-9fd3-14518c7278fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:25:27.010806 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d158fa1f-5622-4e06-9fd3-14518c7278fc admin admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:25:27.011082 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d158fa1f-5622-4e06-9fd3-14518c7278fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:25:27.011345 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d158fa1f-5622-4e06-9fd3-14518c7278fc admin admin] PATH_INFO: `/v3/roles` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:25:27.011978 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d158fa1f-5622-4e06-9fd3-14518c7278fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:25:27.013598 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d158fa1f-5622-4e06-9fd3-14518c7278fc admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:25:27.015520 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-778148f2-6115-4d95-b10c-4d924a588e3e service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:25:27.017032 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-778148f2-6115-4d95-b10c-4d924a588e3e service glance] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:25:27.020551 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:25:27.024646 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1055/2131] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 12:25:26 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 15 12:25:27.029148 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d158fa1f-5622-4e06-9fd3-14518c7278fc admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:25:27.030540 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1077/2132] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 12:25:26 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 205 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 12:25:27.039203 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d158fa1f-5622-4e06-9fd3-14518c7278fc admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:25:27.043762 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36aa9d1-a202-42df-b0de-8db91510860a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:25:27.044360 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36aa9d1-a202-42df-b0de-8db91510860a admin admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:25:27.044634 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36aa9d1-a202-42df-b0de-8db91510860a admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:25:27.044935 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36aa9d1-a202-42df-b0de-8db91510860a admin admin] PATH_INFO: `/v3/roles` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:25:27.045566 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e36aa9d1-a202-42df-b0de-8db91510860a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:25:27.047130 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e36aa9d1-a202-42df-b0de-8db91510860a admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:25:27.061426 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1056/2133] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 12:25:27 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 15 12:25:27.065406 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778148f2-6115-4d95-b10c-4d924a588e3e service glance] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:25:27.076494 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778148f2-6115-4d95-b10c-4d924a588e3e service glance] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:25:27.084338 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8b65c8-9e6d-470e-b105-e62691105c99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:25:27.085039 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8b65c8-9e6d-470e-b105-e62691105c99 admin admin] REQUEST_METHOD: `PUT` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:25:27.085317 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8b65c8-9e6d-470e-b105-e62691105c99 admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:25:27.085650 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8b65c8-9e6d-470e-b105-e62691105c99 admin admin] PATH_INFO: `/v3/projects/0d44768bacb443dfab7c3754ff186a95/users/b1d41e8bc215406ca6dfe3de220664ac/roles/0679d6628a25438d90dc4bdc0401eda2` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:25:27.103424 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8b65c8-9e6d-470e-b105-e62691105c99 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0d44768bacb443dfab7c3754ff186a95, user_id=b1d41e8bc215406ca6dfe3de220664ac, role_id=0679d6628a25438d90dc4bdc0401eda2)` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:25:27.106437 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8b65c8-9e6d-470e-b105-e62691105c99 admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:25:27.119737 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1078/2134] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 15 12:25:27 2022] PUT /identity/v3/projects/0d44768bacb443dfab7c3754ff186a95/users/b1d41e8bc215406ca6dfe3de220664ac/roles/0679d6628a25438d90dc4bdc0401eda2 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:25:27.130003 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410ddd95-5895-4642-a598-8b98f97881b8 None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:25:27.130421 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410ddd95-5895-4642-a598-8b98f97881b8 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:25:27.130732 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410ddd95-5895-4642-a598-8b98f97881b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:25:27.160920 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-410ddd95-5895-4642-a598-8b98f97881b8 None None] MFA Rules not processed for user `b1d41e8bc215406ca6dfe3de220664ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:25:27.162139 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-410ddd95-5895-4642-a598-8b98f97881b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:25:27.233708 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1057/2135] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 12:25:27 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:25:27.238245 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8b65c8-9e6d-470e-b105-e62691105c99 admin admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:25:27.318628 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8b65c8-9e6d-470e-b105-e62691105c99 admin admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:25:27.338498 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57f451b-5b6b-4cfd-bc7b-5dbc141122b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:25:27.339307 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57f451b-5b6b-4cfd-bc7b-5dbc141122b0 admin admin] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:25:27.339768 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57f451b-5b6b-4cfd-bc7b-5dbc141122b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:25:27.340190 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57f451b-5b6b-4cfd-bc7b-5dbc141122b0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:25:27.341103 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f57f451b-5b6b-4cfd-bc7b-5dbc141122b0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:25:27.343851 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f57f451b-5b6b-4cfd-bc7b-5dbc141122b0 admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:25:27.383002 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1079/2136] 10.222.0.15 () {60 vars in 1153 bytes} [Thu Sep 15 12:25:27 2022] POST /identity/v3/projects => generated 385 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 12:25:27.387547 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410ddd95-5895-4642-a598-8b98f97881b8 None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:25:27.401709 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410ddd95-5895-4642-a598-8b98f97881b8 None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:25:27.487926 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fdb81e7-0976-4b04-919e-cf3b823e8ddb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:25:27.488808 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdb81e7-0976-4b04-919e-cf3b823e8ddb admin admin] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:25:27.488915 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdb81e7-0976-4b04-919e-cf3b823e8ddb admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:25:27.489160 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdb81e7-0976-4b04-919e-cf3b823e8ddb admin admin] PATH_INFO: `/v3/users` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:25:27.489785 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fdb81e7-0976-4b04-919e-cf3b823e8ddb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:25:27.491925 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fdb81e7-0976-4b04-919e-cf3b823e8ddb admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:25:27.540075 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1058/2137] 10.222.0.15 () {60 vars in 1144 bytes} [Thu Sep 15 12:25:27 2022] POST /identity/v3/users => generated 299 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 15 12:25:27.541371 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57f451b-5b6b-4cfd-bc7b-5dbc141122b0 admin admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:25:27.555320 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57f451b-5b6b-4cfd-bc7b-5dbc141122b0 admin admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:25:27.566714 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13479b6a-4cb0-40d8-8ac1-71e8499c1b6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:25:27.568011 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13479b6a-4cb0-40d8-8ac1-71e8499c1b6d admin admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:25:27.568514 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13479b6a-4cb0-40d8-8ac1-71e8499c1b6d admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:25:27.568974 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13479b6a-4cb0-40d8-8ac1-71e8499c1b6d admin admin] PATH_INFO: `/v3/roles` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:25:27.570335 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13479b6a-4cb0-40d8-8ac1-71e8499c1b6d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:25:27.573469 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13479b6a-4cb0-40d8-8ac1-71e8499c1b6d admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:25:27.602737 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1080/2138] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 12:25:27 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 12:25:27.607061 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fdb81e7-0976-4b04-919e-cf3b823e8ddb admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:25:27.615265 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:25:27.620876 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fdb81e7-0976-4b04-919e-cf3b823e8ddb admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:25:27.640401 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848f7cdf-892d-4ad5-97b1-b90ac9f55c34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:25:27.641400 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848f7cdf-892d-4ad5-97b1-b90ac9f55c34 admin admin] REQUEST_METHOD: `PUT` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:25:27.641539 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848f7cdf-892d-4ad5-97b1-b90ac9f55c34 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:25:27.641892 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848f7cdf-892d-4ad5-97b1-b90ac9f55c34 admin admin] PATH_INFO: `/v3/projects/11fd50c8071943ac957897730ce2099c/users/8fb9626f89b347eea70aab31b2f5e6be/roles/9c769b5c82884766b2c510b1f0f1364c` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:25:27.660431 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848f7cdf-892d-4ad5-97b1-b90ac9f55c34 admin admin] RBAC: Authorizing `identity:create_grant(project_id=11fd50c8071943ac957897730ce2099c, user_id=8fb9626f89b347eea70aab31b2f5e6be, role_id=9c769b5c82884766b2c510b1f0f1364c)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:25:27.663438 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848f7cdf-892d-4ad5-97b1-b90ac9f55c34 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:25:27.679753 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1059/2139] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 15 12:25:27 2022] PUT /identity/v3/projects/11fd50c8071943ac957897730ce2099c/users/8fb9626f89b347eea70aab31b2f5e6be/roles/9c769b5c82884766b2c510b1f0f1364c => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 12:25:27.684466 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13479b6a-4cb0-40d8-8ac1-71e8499c1b6d admin admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:25:27.695619 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13479b6a-4cb0-40d8-8ac1-71e8499c1b6d admin admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:25:27.803624 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f638b636-20dd-4cdd-8fbe-c3278169cdca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:25:27.804277 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f638b636-20dd-4cdd-8fbe-c3278169cdca admin admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:25:27.804559 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f638b636-20dd-4cdd-8fbe-c3278169cdca admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:25:27.804860 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f638b636-20dd-4cdd-8fbe-c3278169cdca admin admin] PATH_INFO: `/v3/roles` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:25:27.809045 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f638b636-20dd-4cdd-8fbe-c3278169cdca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:25:27.811204 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f638b636-20dd-4cdd-8fbe-c3278169cdca admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:25:27.828689 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1081/2140] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 12:25:27 2022] GET /identity/v3/roles => generated 1420 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 12:25:27.836376 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848f7cdf-892d-4ad5-97b1-b90ac9f55c34 admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:25:27.846400 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848f7cdf-892d-4ad5-97b1-b90ac9f55c34 admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:25:27.852492 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a331b2-82ae-4ee1-9433-e748243d3bb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:25:27.852492 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a331b2-82ae-4ee1-9433-e748243d3bb5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:25:27.852792 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a331b2-82ae-4ee1-9433-e748243d3bb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:25:27.852879 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a331b2-82ae-4ee1-9433-e748243d3bb5 admin admin] PATH_INFO: `/v3/projects/11fd50c8071943ac957897730ce2099c/users/8fb9626f89b347eea70aab31b2f5e6be/roles/9c769b5c82884766b2c510b1f0f1364c` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:25:27.855416 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a331b2-82ae-4ee1-9433-e748243d3bb5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=11fd50c8071943ac957897730ce2099c, user_id=8fb9626f89b347eea70aab31b2f5e6be, role_id=9c769b5c82884766b2c510b1f0f1364c)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:25:27.858432 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a331b2-82ae-4ee1-9433-e748243d3bb5 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:25:27.871718 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1060/2141] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 15 12:25:27 2022] PUT /identity/v3/projects/11fd50c8071943ac957897730ce2099c/users/8fb9626f89b347eea70aab31b2f5e6be/roles/9c769b5c82884766b2c510b1f0f1364c => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 12:25:27.883106 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c42521-dd44-4b9b-a03e-aad1d7e4ef27 None None] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:25:27.883396 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c42521-dd44-4b9b-a03e-aad1d7e4ef27 None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:25:27.883679 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c42521-dd44-4b9b-a03e-aad1d7e4ef27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:25:27.918544 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.auth.core [None req-80c42521-dd44-4b9b-a03e-aad1d7e4ef27 None None] MFA Rules not processed for user `8fb9626f89b347eea70aab31b2f5e6be`. Rule list: `[]` (Enabled: `True`). {{(pid=81449) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:25:27.920623 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-80c42521-dd44-4b9b-a03e-aad1d7e4ef27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:25:27.971365 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:25:28.047112 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1082/2142] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 12:25:27 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:25:28.063696 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a331b2-82ae-4ee1-9433-e748243d3bb5 admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:25:28.067553 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-f2a331b2-82ae-4ee1-9433-e748243d3bb5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:25:28.148170 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a331b2-82ae-4ee1-9433-e748243d3bb5 admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:25:28.160295 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e803ebf3-9ac3-4dea-9779-7eaeacfa659c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8191d3ccb874529b32f8d0026db629d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '540a32ef88764e8c8d9201ae22f6ddaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:25:28.161072 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e803ebf3-9ac3-4dea-9779-7eaeacfa659c service cinder] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:25:28.161447 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e803ebf3-9ac3-4dea-9779-7eaeacfa659c service cinder] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:25:28.161747 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e803ebf3-9ac3-4dea-9779-7eaeacfa659c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:25:28.173066 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e803ebf3-9ac3-4dea-9779-7eaeacfa659c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:25:28.174509 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e803ebf3-9ac3-4dea-9779-7eaeacfa659c service cinder] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:25:28.192017 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1061/2143] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 12:25:28 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 12:25:28.230736 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c42521-dd44-4b9b-a03e-aad1d7e4ef27 None None] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:25:28.303853 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c42521-dd44-4b9b-a03e-aad1d7e4ef27 None None] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:25:28.311288 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c700784-388e-43b4-b3eb-ab7b63dd0759 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:25:28.312490 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c700784-388e-43b4-b3eb-ab7b63dd0759 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:25:28.312888 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c700784-388e-43b4-b3eb-ab7b63dd0759 admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:25:28.313240 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c700784-388e-43b4-b3eb-ab7b63dd0759 admin admin] PATH_INFO: `/v3/users/b1d41e8bc215406ca6dfe3de220664ac` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:25:28.314663 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c700784-388e-43b4-b3eb-ab7b63dd0759 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b1d41e8bc215406ca6dfe3de220664ac)` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:25:28.316906 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c700784-388e-43b4-b3eb-ab7b63dd0759 admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:25:28.373422 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-5c700784-388e-43b4-b3eb-ab7b63dd0759 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b1d41e8bc215406ca6dfe3de220664ac'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:25:28.381599 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-5c700784-388e-43b4-b3eb-ab7b63dd0759 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b1d41e8bc215406ca6dfe3de220664ac'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:25:28.398257 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-5c700784-388e-43b4-b3eb-ab7b63dd0759 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b1d41e8bc215406ca6dfe3de220664ac'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:25:28.399023 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-5c700784-388e-43b4-b3eb-ab7b63dd0759 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b1d41e8bc215406ca6dfe3de220664ac'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:25:28.407729 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-5c700784-388e-43b4-b3eb-ab7b63dd0759 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b1d41e8bc215406ca6dfe3de220664ac'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:25:28.413646 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1083/2144] 10.222.0.15 () {58 vars in 1224 bytes} [Thu Sep 15 12:25:28 2022] DELETE /identity/v3/users/b1d41e8bc215406ca6dfe3de220664ac => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:25:28.418083 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ed8ddd-fc94-486e-9f12-abec9728c30f None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:25:28.418438 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ed8ddd-fc94-486e-9f12-abec9728c30f None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:25:28.418797 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ed8ddd-fc94-486e-9f12-abec9728c30f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:25:28.439045 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-84ed8ddd-fc94-486e-9f12-abec9728c30f None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:25:28.440417 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-84ed8ddd-fc94-486e-9f12-abec9728c30f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:25:28.510249 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1062/2145] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 12:25:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 12:25:28.520510 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c700784-388e-43b4-b3eb-ab7b63dd0759 admin admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:25:28.522287 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-5c700784-388e-43b4-b3eb-ab7b63dd0759 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:25:28.613234 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:25:28.632661 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c700784-388e-43b4-b3eb-ab7b63dd0759 admin admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:25:28.638816 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddddd151-2316-4601-a5ef-2b48611f786e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfbb4d6809064f9cb9e06626ed7d6814', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '540a32ef88764e8c8d9201ae22f6ddaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:25:28.639560 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddddd151-2316-4601-a5ef-2b48611f786e service neutron] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:25:28.639985 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddddd151-2316-4601-a5ef-2b48611f786e service neutron] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:25:28.640807 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddddd151-2316-4601-a5ef-2b48611f786e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:25:28.657893 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddddd151-2316-4601-a5ef-2b48611f786e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:25:28.659734 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddddd151-2316-4601-a5ef-2b48611f786e service neutron] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:25:28.671565 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1084/2146] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 12:25:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 12:25:28.999986 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ed8ddd-fc94-486e-9f12-abec9728c30f None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:25:29.001546 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-84ed8ddd-fc94-486e-9f12-abec9728c30f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:25:29.025896 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ed8ddd-fc94-486e-9f12-abec9728c30f None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:25:29.031274 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413c34ca-df16-450a-af10-b62498deb034 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:25:29.031923 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413c34ca-df16-450a-af10-b62498deb034 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:25:29.032211 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413c34ca-df16-450a-af10-b62498deb034 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:25:29.032965 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413c34ca-df16-450a-af10-b62498deb034 admin admin] PATH_INFO: `/v3/projects/0d44768bacb443dfab7c3754ff186a95` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:25:29.034118 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413c34ca-df16-450a-af10-b62498deb034 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0d44768bacb443dfab7c3754ff186a95)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:25:29.036307 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413c34ca-df16-450a-af10-b62498deb034 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:25:29.088631 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-413c34ca-df16-450a-af10-b62498deb034 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0d44768bacb443dfab7c3754ff186a95', 'request_id': 'req-413c34ca-df16-450a-af10-b62498deb034'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:25:29.098117 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-413c34ca-df16-450a-af10-b62498deb034 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0d44768bacb443dfab7c3754ff186a95', 'request_id': 'req-413c34ca-df16-450a-af10-b62498deb034'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:25:29.104031 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-413c34ca-df16-450a-af10-b62498deb034 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0d44768bacb443dfab7c3754ff186a95', 'request_id': 'req-413c34ca-df16-450a-af10-b62498deb034'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:25:29.105171 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:25:29.117432 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-413c34ca-df16-450a-af10-b62498deb034 admin admin] The token cache is being invalidate because project 0d44768bacb443dfab7c3754ff186a95 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81448) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 12:25:29.117945 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-413c34ca-df16-450a-af10-b62498deb034 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:25:29.118901 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-413c34ca-df16-450a-af10-b62498deb034 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:25:29.120952 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1063/2147] 10.222.0.15 () {58 vars in 1233 bytes} [Thu Sep 15 12:25:28 2022] DELETE /identity/v3/projects/0d44768bacb443dfab7c3754ff186a95 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:25:29.125078 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddddd151-2316-4601-a5ef-2b48611f786e service neutron] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:25:29.134775 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddddd151-2316-4601-a5ef-2b48611f786e service neutron] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:25:29.195686 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483bc17b-fcbd-4af2-81aa-79a197c8a204 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:25:29.196320 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483bc17b-fcbd-4af2-81aa-79a197c8a204 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:25:29.196596 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483bc17b-fcbd-4af2-81aa-79a197c8a204 admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:25:29.196898 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483bc17b-fcbd-4af2-81aa-79a197c8a204 admin admin] PATH_INFO: `/v3/users/8fb9626f89b347eea70aab31b2f5e6be` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:25:29.198046 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483bc17b-fcbd-4af2-81aa-79a197c8a204 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8fb9626f89b347eea70aab31b2f5e6be)` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:25:29.199941 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-483bc17b-fcbd-4af2-81aa-79a197c8a204 admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:25:29.253762 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-483bc17b-fcbd-4af2-81aa-79a197c8a204 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8fb9626f89b347eea70aab31b2f5e6be'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:25:29.262477 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-483bc17b-fcbd-4af2-81aa-79a197c8a204 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8fb9626f89b347eea70aab31b2f5e6be'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:25:29.276127 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-483bc17b-fcbd-4af2-81aa-79a197c8a204 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8fb9626f89b347eea70aab31b2f5e6be'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:25:29.276884 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-483bc17b-fcbd-4af2-81aa-79a197c8a204 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8fb9626f89b347eea70aab31b2f5e6be'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:25:29.284251 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-483bc17b-fcbd-4af2-81aa-79a197c8a204 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8fb9626f89b347eea70aab31b2f5e6be'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:25:29.291445 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1085/2148] 10.222.0.15 () {58 vars in 1224 bytes} [Thu Sep 15 12:25:29 2022] DELETE /identity/v3/users/8fb9626f89b347eea70aab31b2f5e6be => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:25:30.105576 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413c34ca-df16-450a-af10-b62498deb034 admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:25:30.107225 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-413c34ca-df16-450a-af10-b62498deb034 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:25:30.122741 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413c34ca-df16-450a-af10-b62498deb034 admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:25:30.206182 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b03aa67-210c-4356-aec7-f52f26c7a6b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:25:30.206689 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b03aa67-210c-4356-aec7-f52f26c7a6b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:25:30.207127 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b03aa67-210c-4356-aec7-f52f26c7a6b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:25:30.207409 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b03aa67-210c-4356-aec7-f52f26c7a6b7 admin admin] PATH_INFO: `/v3/projects/11fd50c8071943ac957897730ce2099c` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:25:30.208560 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b03aa67-210c-4356-aec7-f52f26c7a6b7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=11fd50c8071943ac957897730ce2099c)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:25:30.210548 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b03aa67-210c-4356-aec7-f52f26c7a6b7 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:25:30.259384 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-2b03aa67-210c-4356-aec7-f52f26c7a6b7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '11fd50c8071943ac957897730ce2099c', 'request_id': 'req-2b03aa67-210c-4356-aec7-f52f26c7a6b7'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:25:30.267474 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-2b03aa67-210c-4356-aec7-f52f26c7a6b7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '11fd50c8071943ac957897730ce2099c', 'request_id': 'req-2b03aa67-210c-4356-aec7-f52f26c7a6b7'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:25:30.273322 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-2b03aa67-210c-4356-aec7-f52f26c7a6b7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '11fd50c8071943ac957897730ce2099c', 'request_id': 'req-2b03aa67-210c-4356-aec7-f52f26c7a6b7'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:25:30.286223 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-2b03aa67-210c-4356-aec7-f52f26c7a6b7 admin admin] The token cache is being invalidate because project 11fd50c8071943ac957897730ce2099c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81448) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 12:25:30.286846 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-2b03aa67-210c-4356-aec7-f52f26c7a6b7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:25:30.287557 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-2b03aa67-210c-4356-aec7-f52f26c7a6b7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:25:30.290341 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1064/2149] 10.222.0.15 () {58 vars in 1233 bytes} [Thu Sep 15 12:25:30 2022] DELETE /identity/v3/projects/11fd50c8071943ac957897730ce2099c => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 12:25:30.296244 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce2ca38-cb9f-4eba-87df-3000f9740fc7 None None] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:25:30.296806 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce2ca38-cb9f-4eba-87df-3000f9740fc7 None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:25:30.296998 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce2ca38-cb9f-4eba-87df-3000f9740fc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:25:30.318259 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.auth.core [None req-2ce2ca38-cb9f-4eba-87df-3000f9740fc7 None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81449) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:25:30.319589 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-2ce2ca38-cb9f-4eba-87df-3000f9740fc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:25:30.384785 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1086/2150] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 12:25:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:25:30.393463 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796633ac-d1a2-4211-8b0d-f2100c75d8af None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:25:30.393797 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796633ac-d1a2-4211-8b0d-f2100c75d8af None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:25:30.394115 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796633ac-d1a2-4211-8b0d-f2100c75d8af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:25:30.415404 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-796633ac-d1a2-4211-8b0d-f2100c75d8af None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:25:30.417465 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-796633ac-d1a2-4211-8b0d-f2100c75d8af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:25:30.428549 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1065/2151] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 12:25:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 12:25:30.430105 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce2ca38-cb9f-4eba-87df-3000f9740fc7 None None] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:25:30.449824 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce2ca38-cb9f-4eba-87df-3000f9740fc7 None None] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:25:30.456938 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70eb8c2e-150d-4a91-b1b0-e93c46a38883 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:25:30.457657 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70eb8c2e-150d-4a91-b1b0-e93c46a38883 admin admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:25:30.458007 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70eb8c2e-150d-4a91-b1b0-e93c46a38883 admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:25:30.458341 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70eb8c2e-150d-4a91-b1b0-e93c46a38883 admin admin] PATH_INFO: `/v3/domains` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:25:30.459094 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70eb8c2e-150d-4a91-b1b0-e93c46a38883 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81449) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:25:30.459438 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70eb8c2e-150d-4a91-b1b0-e93c46a38883 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:25:30.461085 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70eb8c2e-150d-4a91-b1b0-e93c46a38883 admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:25:30.468986 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:25:30.475710 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1087/2152] 10.222.0.15 () {58 vars in 1153 bytes} [Thu Sep 15 12:25:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 12:25:30.480527 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796633ac-d1a2-4211-8b0d-f2100c75d8af None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:25:30.490759 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796633ac-d1a2-4211-8b0d-f2100c75d8af None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:25:30.495835 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b25aeb7-dc91-42df-bcc1-92370ae27572 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:25:30.496535 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b25aeb7-dc91-42df-bcc1-92370ae27572 admin admin] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:25:30.496786 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b25aeb7-dc91-42df-bcc1-92370ae27572 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:25:30.497226 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b25aeb7-dc91-42df-bcc1-92370ae27572 admin admin] PATH_INFO: `/v3/projects` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:25:30.497906 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b25aeb7-dc91-42df-bcc1-92370ae27572 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:25:30.500119 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b25aeb7-dc91-42df-bcc1-92370ae27572 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:25:30.540061 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1066/2153] 10.222.0.15 () {60 vars in 1153 bytes} [Thu Sep 15 12:25:30 2022] POST /identity/v3/projects => generated 401 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 12:25:30.544864 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70eb8c2e-150d-4a91-b1b0-e93c46a38883 admin admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:25:30.551464 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b25aeb7-dc91-42df-bcc1-92370ae27572 admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:25:30.554424 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-4b25aeb7-dc91-42df-bcc1-92370ae27572 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:25:30.555987 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70eb8c2e-150d-4a91-b1b0-e93c46a38883 admin admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:25:30.643937 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771733cd-e7d6-424f-8052-b7bac351f97d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:25:30.645214 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771733cd-e7d6-424f-8052-b7bac351f97d admin admin] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:25:30.645602 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771733cd-e7d6-424f-8052-b7bac351f97d admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:25:30.645977 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771733cd-e7d6-424f-8052-b7bac351f97d admin admin] PATH_INFO: `/v3/users` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:25:30.646830 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771733cd-e7d6-424f-8052-b7bac351f97d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:25:30.648878 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771733cd-e7d6-424f-8052-b7bac351f97d admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:25:30.656516 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:25:30.670636 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b25aeb7-dc91-42df-bcc1-92370ae27572 admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:25:30.676264 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1088/2154] 10.222.0.15 () {60 vars in 1144 bytes} [Thu Sep 15 12:25:30 2022] POST /identity/v3/users => generated 301 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 12:25:30.680339 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9316a0ce-8f9f-4387-932a-299b8b655f6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:25:30.680685 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771733cd-e7d6-424f-8052-b7bac351f97d admin admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:25:30.681704 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9316a0ce-8f9f-4387-932a-299b8b655f6b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:25:30.681991 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9316a0ce-8f9f-4387-932a-299b8b655f6b admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:25:30.682382 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9316a0ce-8f9f-4387-932a-299b8b655f6b admin admin] PATH_INFO: `/v3/users/1e57e16f2b2d47d38f7be8bb48486c2f` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:25:30.684635 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9316a0ce-8f9f-4387-932a-299b8b655f6b admin admin] RBAC: Authorizing `identity:delete_user(user_id=1e57e16f2b2d47d38f7be8bb48486c2f)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:25:30.686754 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9316a0ce-8f9f-4387-932a-299b8b655f6b admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:25:30.700105 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771733cd-e7d6-424f-8052-b7bac351f97d admin admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:25:30.705334 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9f49fd-b6b9-4669-806f-aaa0a4c0dff3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:25:30.706063 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9f49fd-b6b9-4669-806f-aaa0a4c0dff3 admin admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:25:30.706433 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9f49fd-b6b9-4669-806f-aaa0a4c0dff3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:25:30.706828 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9f49fd-b6b9-4669-806f-aaa0a4c0dff3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:25:30.707552 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f9f49fd-b6b9-4669-806f-aaa0a4c0dff3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:25:30.709324 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f9f49fd-b6b9-4669-806f-aaa0a4c0dff3 admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:25:30.725197 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1089/2155] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 12:25:30 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 15 12:25:30.729315 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9f49fd-b6b9-4669-806f-aaa0a4c0dff3 admin admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:25:30.746392 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9f49fd-b6b9-4669-806f-aaa0a4c0dff3 admin admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:25:30.753622 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa35394-37c1-4b37-b0bb-c5e611cb7aa3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:25:30.754459 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa35394-37c1-4b37-b0bb-c5e611cb7aa3 admin admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:25:30.755073 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa35394-37c1-4b37-b0bb-c5e611cb7aa3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:25:30.755411 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa35394-37c1-4b37-b0bb-c5e611cb7aa3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:25:30.756301 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa35394-37c1-4b37-b0bb-c5e611cb7aa3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:25:30.759489 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa35394-37c1-4b37-b0bb-c5e611cb7aa3 admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:25:30.786741 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1090/2156] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 12:25:30 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 12:25:30.790478 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-9316a0ce-8f9f-4387-932a-299b8b655f6b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1e57e16f2b2d47d38f7be8bb48486c2f'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:25:30.790823 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa35394-37c1-4b37-b0bb-c5e611cb7aa3 admin admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:25:30.801785 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa35394-37c1-4b37-b0bb-c5e611cb7aa3 admin admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:25:30.806667 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2b42ab-5da6-4575-8434-57112efc8c94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:25:30.807915 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2b42ab-5da6-4575-8434-57112efc8c94 admin admin] REQUEST_METHOD: `PUT` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:25:30.808201 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2b42ab-5da6-4575-8434-57112efc8c94 admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:25:30.808657 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2b42ab-5da6-4575-8434-57112efc8c94 admin admin] PATH_INFO: `/v3/projects/d82939b2da294a10bcd5e181fd7e9cb0/users/ee4eebde51f0441a8083e1b0c3c9eed2/roles/0679d6628a25438d90dc4bdc0401eda2` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:25:30.811698 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-9316a0ce-8f9f-4387-932a-299b8b655f6b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1e57e16f2b2d47d38f7be8bb48486c2f'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:25:30.825306 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-9316a0ce-8f9f-4387-932a-299b8b655f6b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1e57e16f2b2d47d38f7be8bb48486c2f'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:25:30.826344 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-9316a0ce-8f9f-4387-932a-299b8b655f6b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1e57e16f2b2d47d38f7be8bb48486c2f'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:25:30.828580 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2b42ab-5da6-4575-8434-57112efc8c94 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d82939b2da294a10bcd5e181fd7e9cb0, user_id=ee4eebde51f0441a8083e1b0c3c9eed2, role_id=0679d6628a25438d90dc4bdc0401eda2)` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:25:30.831415 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2b42ab-5da6-4575-8434-57112efc8c94 admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:25:30.848830 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-9316a0ce-8f9f-4387-932a-299b8b655f6b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1e57e16f2b2d47d38f7be8bb48486c2f'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:25:30.857469 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1067/2158] 10.222.0.15 () {58 vars in 1224 bytes} [Thu Sep 15 12:25:30 2022] DELETE /identity/v3/users/1e57e16f2b2d47d38f7be8bb48486c2f => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 12:25:30.860975 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1091/2158] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 15 12:25:30 2022] PUT /identity/v3/projects/d82939b2da294a10bcd5e181fd7e9cb0/users/ee4eebde51f0441a8083e1b0c3c9eed2/roles/0679d6628a25438d90dc4bdc0401eda2 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 12:25:30.861973 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841513a2-49b9-4256-b83f-741e0b034eba None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:25:30.862216 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841513a2-49b9-4256-b83f-741e0b034eba None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:25:30.862452 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce978ce-9d9a-44a9-99c6-5d826ab2f439 None None] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:25:30.862582 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841513a2-49b9-4256-b83f-741e0b034eba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:25:30.862739 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce978ce-9d9a-44a9-99c6-5d826ab2f439 None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:25:30.863019 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce978ce-9d9a-44a9-99c6-5d826ab2f439 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:25:30.885097 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-841513a2-49b9-4256-b83f-741e0b034eba None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:25:30.886322 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-841513a2-49b9-4256-b83f-741e0b034eba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:25:30.897558 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.auth.core [None req-fce978ce-9d9a-44a9-99c6-5d826ab2f439 None None] MFA Rules not processed for user `ee4eebde51f0441a8083e1b0c3c9eed2`. Rule list: `[]` (Enabled: `True`). {{(pid=81449) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:25:30.898955 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-fce978ce-9d9a-44a9-99c6-5d826ab2f439 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:25:30.976175 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1068/2159] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 12:25:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 12:25:30.985887 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841513a2-49b9-4256-b83f-741e0b034eba None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:25:30.988847 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-841513a2-49b9-4256-b83f-741e0b034eba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:25:31.010433 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1092/2160] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 12:25:30 2022] POST /identity/v3/auth/tokens => generated 3278 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:25:31.014465 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce978ce-9d9a-44a9-99c6-5d826ab2f439 None None] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:25:31.020245 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-fce978ce-9d9a-44a9-99c6-5d826ab2f439 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:25:31.046293 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce978ce-9d9a-44a9-99c6-5d826ab2f439 None None] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:25:31.051857 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6835ac-d144-41da-aad5-9b03ace121a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:25:31.052596 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6835ac-d144-41da-aad5-9b03ace121a5 admin admin] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:25:31.053002 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6835ac-d144-41da-aad5-9b03ace121a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:25:31.053340 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6835ac-d144-41da-aad5-9b03ace121a5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:25:31.054037 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6835ac-d144-41da-aad5-9b03ace121a5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:25:31.056035 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6835ac-d144-41da-aad5-9b03ace121a5 admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:25:31.071471 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841513a2-49b9-4256-b83f-741e0b034eba None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:25:31.075586 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c274ed65-4ddd-4b3f-b62e-2b5578b14bb2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfbb4d6809064f9cb9e06626ed7d6814', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '540a32ef88764e8c8d9201ae22f6ddaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:25:31.079658 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c274ed65-4ddd-4b3f-b62e-2b5578b14bb2 service neutron] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:25:31.079958 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c274ed65-4ddd-4b3f-b62e-2b5578b14bb2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:25:31.080240 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c274ed65-4ddd-4b3f-b62e-2b5578b14bb2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:25:31.095570 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1093/2161] 10.222.0.15 () {60 vars in 1153 bytes} [Thu Sep 15 12:25:31 2022] POST /identity/v3/projects => generated 399 bytes in 81 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 15 12:25:31.096015 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c274ed65-4ddd-4b3f-b62e-2b5578b14bb2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:25:31.098209 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c274ed65-4ddd-4b3f-b62e-2b5578b14bb2 service neutron] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:25:31.099905 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6835ac-d144-41da-aad5-9b03ace121a5 admin admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:25:31.110585 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1069/2162] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 12:25:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 12:25:31.112210 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:25:31.179148 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6835ac-d144-41da-aad5-9b03ace121a5 admin admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:25:31.190254 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3eda73e-4898-4bbd-85a3-618d814e6909 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:25:31.191280 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3eda73e-4898-4bbd-85a3-618d814e6909 admin admin] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:25:31.191817 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3eda73e-4898-4bbd-85a3-618d814e6909 admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:25:31.192300 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3eda73e-4898-4bbd-85a3-618d814e6909 admin admin] PATH_INFO: `/v3/users` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:25:31.193393 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3eda73e-4898-4bbd-85a3-618d814e6909 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:25:31.196737 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3eda73e-4898-4bbd-85a3-618d814e6909 admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:25:31.223951 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1094/2163] 10.222.0.15 () {60 vars in 1144 bytes} [Thu Sep 15 12:25:31 2022] POST /identity/v3/users => generated 306 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 12:25:31.228100 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c274ed65-4ddd-4b3f-b62e-2b5578b14bb2 service neutron] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:25:31.248493 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c274ed65-4ddd-4b3f-b62e-2b5578b14bb2 service neutron] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:25:31.255664 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe854523-8f66-4204-b84f-48a42dff00ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:25:31.256373 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe854523-8f66-4204-b84f-48a42dff00ad admin admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:25:31.256726 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe854523-8f66-4204-b84f-48a42dff00ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:25:31.257096 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe854523-8f66-4204-b84f-48a42dff00ad admin admin] PATH_INFO: `/v3/roles` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:25:31.257809 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe854523-8f66-4204-b84f-48a42dff00ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:25:31.259573 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe854523-8f66-4204-b84f-48a42dff00ad admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:25:31.276707 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1070/2164] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 12:25:31 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 12:25:31.281986 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3eda73e-4898-4bbd-85a3-618d814e6909 admin admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:25:31.294718 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3eda73e-4898-4bbd-85a3-618d814e6909 admin admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:25:31.315657 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d7110a-a153-48a6-86bc-a5f50c464aaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:25:31.318304 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d7110a-a153-48a6-86bc-a5f50c464aaf admin admin] REQUEST_METHOD: `PUT` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:25:31.318835 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d7110a-a153-48a6-86bc-a5f50c464aaf admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:25:31.319267 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d7110a-a153-48a6-86bc-a5f50c464aaf admin admin] PATH_INFO: `/v3/projects/669bd636e9914855b21c1764cddd7038/users/7d89d40d67814e6197b7ad21647df5d6/roles/9c769b5c82884766b2c510b1f0f1364c` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:25:31.343640 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d7110a-a153-48a6-86bc-a5f50c464aaf admin admin] RBAC: Authorizing `identity:create_grant(project_id=669bd636e9914855b21c1764cddd7038, user_id=7d89d40d67814e6197b7ad21647df5d6, role_id=9c769b5c82884766b2c510b1f0f1364c)` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:25:31.352785 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d7110a-a153-48a6-86bc-a5f50c464aaf admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:25:31.370577 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1095/2165] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 15 12:25:31 2022] PUT /identity/v3/projects/669bd636e9914855b21c1764cddd7038/users/7d89d40d67814e6197b7ad21647df5d6/roles/9c769b5c82884766b2c510b1f0f1364c => generated 0 bytes in 90 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 12:25:31.374323 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe854523-8f66-4204-b84f-48a42dff00ad admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:25:31.388233 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe854523-8f66-4204-b84f-48a42dff00ad admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:25:31.393983 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:25:31.487516 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33508c25-c81a-4cbf-a470-5602f794a91c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:25:31.488141 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33508c25-c81a-4cbf-a470-5602f794a91c admin admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:25:31.488483 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33508c25-c81a-4cbf-a470-5602f794a91c admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:25:31.488778 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33508c25-c81a-4cbf-a470-5602f794a91c admin admin] PATH_INFO: `/v3/roles` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:25:31.489623 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33508c25-c81a-4cbf-a470-5602f794a91c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:25:31.491895 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33508c25-c81a-4cbf-a470-5602f794a91c admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:25:31.513030 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1071/2166] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 12:25:31 2022] GET /identity/v3/roles => generated 1420 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 12:25:31.515900 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d7110a-a153-48a6-86bc-a5f50c464aaf admin admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:25:31.526578 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d7110a-a153-48a6-86bc-a5f50c464aaf admin admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:25:31.549290 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d7c662-7d35-4325-82b9-415ad90d2f49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:25:31.550937 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d7c662-7d35-4325-82b9-415ad90d2f49 admin admin] REQUEST_METHOD: `PUT` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:25:31.551447 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d7c662-7d35-4325-82b9-415ad90d2f49 admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:25:31.553116 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d7c662-7d35-4325-82b9-415ad90d2f49 admin admin] PATH_INFO: `/v3/projects/669bd636e9914855b21c1764cddd7038/users/7d89d40d67814e6197b7ad21647df5d6/roles/9c769b5c82884766b2c510b1f0f1364c` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:25:31.559884 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d7c662-7d35-4325-82b9-415ad90d2f49 admin admin] RBAC: Authorizing `identity:create_grant(project_id=669bd636e9914855b21c1764cddd7038, user_id=7d89d40d67814e6197b7ad21647df5d6, role_id=9c769b5c82884766b2c510b1f0f1364c)` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:25:31.565186 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d7c662-7d35-4325-82b9-415ad90d2f49 admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:25:31.594220 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1096/2167] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 15 12:25:31 2022] PUT /identity/v3/projects/669bd636e9914855b21c1764cddd7038/users/7d89d40d67814e6197b7ad21647df5d6/roles/9c769b5c82884766b2c510b1f0f1364c => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:25:31.606861 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90429069-bdf7-41c7-bdfc-9c734ff56e1e None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:25:31.607410 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90429069-bdf7-41c7-bdfc-9c734ff56e1e None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:25:31.607894 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90429069-bdf7-41c7-bdfc-9c734ff56e1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:25:31.650195 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-90429069-bdf7-41c7-bdfc-9c734ff56e1e None None] MFA Rules not processed for user `7d89d40d67814e6197b7ad21647df5d6`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:25:31.651865 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-90429069-bdf7-41c7-bdfc-9c734ff56e1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:25:31.723692 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d7c662-7d35-4325-82b9-415ad90d2f49 admin admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:25:31.726177 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-d8d7c662-7d35-4325-82b9-415ad90d2f49 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:25:31.746937 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1072/2168] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 12:25:31 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:25:31.759424 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90429069-bdf7-41c7-bdfc-9c734ff56e1e None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:25:31.761596 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-90429069-bdf7-41c7-bdfc-9c734ff56e1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:25:31.827830 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d7c662-7d35-4325-82b9-415ad90d2f49 admin admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:25:31.833824 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c461bb1-4919-4e9e-b83d-badb38d85894 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:25:31.834784 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c461bb1-4919-4e9e-b83d-badb38d85894 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:25:31.835185 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c461bb1-4919-4e9e-b83d-badb38d85894 admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:25:31.835539 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c461bb1-4919-4e9e-b83d-badb38d85894 admin admin] PATH_INFO: `/v3/projects/506b60c4fe2b4e3faa9c5ec182b7fcf9` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:25:31.836982 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c461bb1-4919-4e9e-b83d-badb38d85894 admin admin] RBAC: Authorizing `identity:delete_project(project_id=506b60c4fe2b4e3faa9c5ec182b7fcf9)` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:25:31.839882 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c461bb1-4919-4e9e-b83d-badb38d85894 admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:25:31.894847 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90429069-bdf7-41c7-bdfc-9c734ff56e1e None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:25:31.906821 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-7c461bb1-4919-4e9e-b83d-badb38d85894 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '506b60c4fe2b4e3faa9c5ec182b7fcf9', 'request_id': 'req-7c461bb1-4919-4e9e-b83d-badb38d85894'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:25:31.915280 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-7c461bb1-4919-4e9e-b83d-badb38d85894 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '506b60c4fe2b4e3faa9c5ec182b7fcf9', 'request_id': 'req-7c461bb1-4919-4e9e-b83d-badb38d85894'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:25:31.925602 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=87,keystone:INSERT=6,keystone:DELETE=7 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:25:31.930364 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-7c461bb1-4919-4e9e-b83d-badb38d85894 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '506b60c4fe2b4e3faa9c5ec182b7fcf9', 'request_id': 'req-7c461bb1-4919-4e9e-b83d-badb38d85894'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:25:31.964318 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:25:31.967895 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-7c461bb1-4919-4e9e-b83d-badb38d85894 admin admin] The token cache is being invalidate because project 506b60c4fe2b4e3faa9c5ec182b7fcf9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81449) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 12:25:31.968790 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-7c461bb1-4919-4e9e-b83d-badb38d85894 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:25:31.969071 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-7c461bb1-4919-4e9e-b83d-badb38d85894 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:25:31.973809 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1097/2169] 10.222.0.15 () {58 vars in 1233 bytes} [Thu Sep 15 12:25:31 2022] DELETE /identity/v3/projects/506b60c4fe2b4e3faa9c5ec182b7fcf9 => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 12:25:31.996052 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91590aba-62b4-46a9-8811-65384cb471fc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8191d3ccb874529b32f8d0026db629d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '540a32ef88764e8c8d9201ae22f6ddaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:25:31.997027 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91590aba-62b4-46a9-8811-65384cb471fc service cinder] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:25:31.997412 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91590aba-62b4-46a9-8811-65384cb471fc service cinder] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:25:31.997721 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91590aba-62b4-46a9-8811-65384cb471fc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:25:32.000114 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-91590aba-62b4-46a9-8811-65384cb471fc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:25:32.032737 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c461bb1-4919-4e9e-b83d-badb38d85894 admin admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:25:32.040388 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-7c461bb1-4919-4e9e-b83d-badb38d85894 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:25:32.094966 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91590aba-62b4-46a9-8811-65384cb471fc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:25:32.097271 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91590aba-62b4-46a9-8811-65384cb471fc service cinder] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:25:32.111646 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1073/2170] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 12:25:31 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 12:25:32.153067 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c461bb1-4919-4e9e-b83d-badb38d85894 admin admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:25:32.162722 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeda22d8-cb51-4cc1-8c4d-96bd8ba03a8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:25:32.162722 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeda22d8-cb51-4cc1-8c4d-96bd8ba03a8d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:25:32.162722 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeda22d8-cb51-4cc1-8c4d-96bd8ba03a8d admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:25:32.162722 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeda22d8-cb51-4cc1-8c4d-96bd8ba03a8d admin admin] PATH_INFO: `/v3/users/b5c77cfb2ad8404f9d0dff4f9bdb98ba` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:25:32.163334 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeda22d8-cb51-4cc1-8c4d-96bd8ba03a8d admin admin] RBAC: Authorizing `identity:delete_user(user_id=b5c77cfb2ad8404f9d0dff4f9bdb98ba)` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:25:32.163818 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeda22d8-cb51-4cc1-8c4d-96bd8ba03a8d admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:25:32.220103 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91590aba-62b4-46a9-8811-65384cb471fc service cinder] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:25:32.222149 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-91590aba-62b4-46a9-8811-65384cb471fc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:25:32.227837 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-eeda22d8-cb51-4cc1-8c4d-96bd8ba03a8d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b5c77cfb2ad8404f9d0dff4f9bdb98ba'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:25:32.233611 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91590aba-62b4-46a9-8811-65384cb471fc service cinder] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:25:32.239312 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-eeda22d8-cb51-4cc1-8c4d-96bd8ba03a8d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b5c77cfb2ad8404f9d0dff4f9bdb98ba'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:25:32.245369 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215dccbf-7281-4126-91ca-8fc8641fbefc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8191d3ccb874529b32f8d0026db629d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '540a32ef88764e8c8d9201ae22f6ddaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:25:32.246371 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215dccbf-7281-4126-91ca-8fc8641fbefc service cinder] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:25:32.246821 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215dccbf-7281-4126-91ca-8fc8641fbefc service cinder] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:25:32.247115 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215dccbf-7281-4126-91ca-8fc8641fbefc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:25:32.249036 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-215dccbf-7281-4126-91ca-8fc8641fbefc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:25:32.252159 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-eeda22d8-cb51-4cc1-8c4d-96bd8ba03a8d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b5c77cfb2ad8404f9d0dff4f9bdb98ba'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:25:32.253254 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-eeda22d8-cb51-4cc1-8c4d-96bd8ba03a8d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b5c77cfb2ad8404f9d0dff4f9bdb98ba'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:25:32.261181 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-eeda22d8-cb51-4cc1-8c4d-96bd8ba03a8d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b5c77cfb2ad8404f9d0dff4f9bdb98ba'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:25:32.267710 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1098/2171] 10.222.0.15 () {58 vars in 1224 bytes} [Thu Sep 15 12:25:32 2022] DELETE /identity/v3/users/b5c77cfb2ad8404f9d0dff4f9bdb98ba => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:25:32.319054 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215dccbf-7281-4126-91ca-8fc8641fbefc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:25:32.320810 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215dccbf-7281-4126-91ca-8fc8641fbefc service cinder] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:25:32.332135 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1074/2172] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 12:25:32 2022] GET /identity/v3/auth/tokens => generated 3278 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 12:25:32.483121 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeda22d8-cb51-4cc1-8c4d-96bd8ba03a8d admin admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:25:32.485398 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-eeda22d8-cb51-4cc1-8c4d-96bd8ba03a8d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:25:32.580515 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeda22d8-cb51-4cc1-8c4d-96bd8ba03a8d admin admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:25:32.585500 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f90d7c-c77f-4f89-9a26-e078da09213d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:25:32.586136 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f90d7c-c77f-4f89-9a26-e078da09213d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:25:32.586411 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f90d7c-c77f-4f89-9a26-e078da09213d admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:25:32.586750 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f90d7c-c77f-4f89-9a26-e078da09213d admin admin] PATH_INFO: `/v3/projects/705a7022acf04881bed0e40d7a491dd4` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:25:32.588331 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f90d7c-c77f-4f89-9a26-e078da09213d admin admin] RBAC: Authorizing `identity:delete_project(project_id=705a7022acf04881bed0e40d7a491dd4)` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:25:32.590152 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f90d7c-c77f-4f89-9a26-e078da09213d admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:25:32.603427 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:25:32.658227 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-f6f90d7c-c77f-4f89-9a26-e078da09213d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '705a7022acf04881bed0e40d7a491dd4', 'request_id': 'req-f6f90d7c-c77f-4f89-9a26-e078da09213d'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:25:32.667553 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-f6f90d7c-c77f-4f89-9a26-e078da09213d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '705a7022acf04881bed0e40d7a491dd4', 'request_id': 'req-f6f90d7c-c77f-4f89-9a26-e078da09213d'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:25:32.674647 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-f6f90d7c-c77f-4f89-9a26-e078da09213d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '705a7022acf04881bed0e40d7a491dd4', 'request_id': 'req-f6f90d7c-c77f-4f89-9a26-e078da09213d'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:25:32.700207 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-f6f90d7c-c77f-4f89-9a26-e078da09213d admin admin] The token cache is being invalidate because project 705a7022acf04881bed0e40d7a491dd4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81449) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 12:25:32.700207 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-f6f90d7c-c77f-4f89-9a26-e078da09213d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:25:32.700207 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-f6f90d7c-c77f-4f89-9a26-e078da09213d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:25:32.700207 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1099/2173] 10.222.0.15 () {58 vars in 1233 bytes} [Thu Sep 15 12:25:32 2022] DELETE /identity/v3/projects/705a7022acf04881bed0e40d7a491dd4 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:25:32.700514 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4d07fc-37b7-45da-a18c-87f19b327358 None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:25:32.700514 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4d07fc-37b7-45da-a18c-87f19b327358 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:25:32.700514 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4d07fc-37b7-45da-a18c-87f19b327358 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:25:32.720481 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-9b4d07fc-37b7-45da-a18c-87f19b327358 None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:25:32.721732 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-9b4d07fc-37b7-45da-a18c-87f19b327358 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:25:32.787500 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1075/2174] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 12:25:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:25:32.795521 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26f3d24-e663-450b-a412-47b820b28828 None None] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:25:32.795807 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26f3d24-e663-450b-a412-47b820b28828 None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:25:32.796082 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26f3d24-e663-450b-a412-47b820b28828 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:25:32.815998 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.auth.core [None req-b26f3d24-e663-450b-a412-47b820b28828 None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81449) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:25:32.817239 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-b26f3d24-e663-450b-a412-47b820b28828 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:25:32.825649 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1100/2175] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 12:25:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:25:32.829746 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4d07fc-37b7-45da-a18c-87f19b327358 None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:25:32.839482 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4d07fc-37b7-45da-a18c-87f19b327358 None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:25:32.844361 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49331f2a-e703-4cfd-b61a-7afb6abc1362 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:25:32.845328 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49331f2a-e703-4cfd-b61a-7afb6abc1362 admin admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:25:32.845604 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49331f2a-e703-4cfd-b61a-7afb6abc1362 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:25:32.845870 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49331f2a-e703-4cfd-b61a-7afb6abc1362 admin admin] PATH_INFO: `/v3/domains` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:25:32.846508 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49331f2a-e703-4cfd-b61a-7afb6abc1362 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:25:32.846803 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49331f2a-e703-4cfd-b61a-7afb6abc1362 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:25:32.848405 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49331f2a-e703-4cfd-b61a-7afb6abc1362 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:25:32.860539 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1076/2176] 10.222.0.15 () {58 vars in 1153 bytes} [Thu Sep 15 12:25:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:25:32.864495 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26f3d24-e663-450b-a412-47b820b28828 None None] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:25:32.874174 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26f3d24-e663-450b-a412-47b820b28828 None None] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:25:32.878916 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14bb2dce-e010-4545-9f04-83f96334a7da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:25:32.879525 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bb2dce-e010-4545-9f04-83f96334a7da admin admin] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:25:32.879782 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bb2dce-e010-4545-9f04-83f96334a7da admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:25:32.880049 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14bb2dce-e010-4545-9f04-83f96334a7da admin admin] PATH_INFO: `/v3/projects` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:25:32.880669 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14bb2dce-e010-4545-9f04-83f96334a7da admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:25:32.882629 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14bb2dce-e010-4545-9f04-83f96334a7da admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:25:32.910124 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1101/2177] 10.222.0.15 () {60 vars in 1153 bytes} [Thu Sep 15 12:25:32 2022] POST /identity/v3/projects => generated 391 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 12:25:32.914059 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49331f2a-e703-4cfd-b61a-7afb6abc1362 admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:25:32.925464 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49331f2a-e703-4cfd-b61a-7afb6abc1362 admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:25:32.947281 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:25:32.989866 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5611e1a-5d88-463e-97ee-93eded829558 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:25:32.990493 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5611e1a-5d88-463e-97ee-93eded829558 admin admin] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:25:32.990851 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5611e1a-5d88-463e-97ee-93eded829558 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:25:32.991197 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5611e1a-5d88-463e-97ee-93eded829558 admin admin] PATH_INFO: `/v3/users` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:25:32.991835 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5611e1a-5d88-463e-97ee-93eded829558 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:25:32.993795 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5611e1a-5d88-463e-97ee-93eded829558 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:25:33.016532 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1077/2178] 10.222.0.15 () {60 vars in 1144 bytes} [Thu Sep 15 12:25:32 2022] POST /identity/v3/users => generated 296 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 15 12:25:33.020875 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14bb2dce-e010-4545-9f04-83f96334a7da admin admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:25:33.031082 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14bb2dce-e010-4545-9f04-83f96334a7da admin admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:25:33.035647 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f3541d-5880-4347-962b-e300d60010de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:25:33.036247 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f3541d-5880-4347-962b-e300d60010de admin admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:25:33.036544 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f3541d-5880-4347-962b-e300d60010de admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:25:33.036886 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f3541d-5880-4347-962b-e300d60010de admin admin] PATH_INFO: `/v3/roles` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:25:33.037528 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f3541d-5880-4347-962b-e300d60010de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:25:33.039156 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f3541d-5880-4347-962b-e300d60010de admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:25:33.050137 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1102/2179] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 12:25:33 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 12:25:33.054130 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5611e1a-5d88-463e-97ee-93eded829558 admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:25:33.065013 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5611e1a-5d88-463e-97ee-93eded829558 admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:25:33.069666 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e192535-62d3-4bbc-bf25-3b6bdd64acb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:25:33.070276 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e192535-62d3-4bbc-bf25-3b6bdd64acb4 admin admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:25:33.070552 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e192535-62d3-4bbc-bf25-3b6bdd64acb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:25:33.070819 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e192535-62d3-4bbc-bf25-3b6bdd64acb4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:25:33.071451 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e192535-62d3-4bbc-bf25-3b6bdd64acb4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:25:33.073265 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e192535-62d3-4bbc-bf25-3b6bdd64acb4 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:25:33.083779 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1078/2180] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 12:25:33 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 12:25:33.087961 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f3541d-5880-4347-962b-e300d60010de admin admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:25:33.097963 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f3541d-5880-4347-962b-e300d60010de admin admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:25:33.102505 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c619686-d36b-44e9-a6cc-631391830942 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:25:33.103658 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c619686-d36b-44e9-a6cc-631391830942 admin admin] REQUEST_METHOD: `PUT` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:25:33.103946 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c619686-d36b-44e9-a6cc-631391830942 admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:25:33.104214 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c619686-d36b-44e9-a6cc-631391830942 admin admin] PATH_INFO: `/v3/projects/22801aa5908b4b098931d2588433e9db/users/20ca976222454c6c898d0e4b170b41fc/roles/0679d6628a25438d90dc4bdc0401eda2` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:25:33.118191 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c619686-d36b-44e9-a6cc-631391830942 admin admin] RBAC: Authorizing `identity:create_grant(project_id=22801aa5908b4b098931d2588433e9db, user_id=20ca976222454c6c898d0e4b170b41fc, role_id=0679d6628a25438d90dc4bdc0401eda2)` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:25:33.121522 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c619686-d36b-44e9-a6cc-631391830942 admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:25:33.137417 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1103/2181] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 15 12:25:33 2022] PUT /identity/v3/projects/22801aa5908b4b098931d2588433e9db/users/20ca976222454c6c898d0e4b170b41fc/roles/0679d6628a25438d90dc4bdc0401eda2 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:25:33.142860 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29090700-dcc5-4afa-8ca7-ad0d1ab5cd14 None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:25:33.143405 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29090700-dcc5-4afa-8ca7-ad0d1ab5cd14 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:25:33.143802 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29090700-dcc5-4afa-8ca7-ad0d1ab5cd14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:25:33.165009 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-29090700-dcc5-4afa-8ca7-ad0d1ab5cd14 None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:25:33.166729 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-29090700-dcc5-4afa-8ca7-ad0d1ab5cd14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:25:33.226713 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1079/2182] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 12:25:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:25:33.237431 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c619686-d36b-44e9-a6cc-631391830942 admin admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:25:33.239378 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-0c619686-d36b-44e9-a6cc-631391830942 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:25:33.298749 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c619686-d36b-44e9-a6cc-631391830942 admin admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:25:33.302393 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d64994-c0b0-4c54-b496-4d9b1bc25650 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfbb4d6809064f9cb9e06626ed7d6814', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '540a32ef88764e8c8d9201ae22f6ddaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:25:33.303180 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d64994-c0b0-4c54-b496-4d9b1bc25650 service neutron] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:25:33.303453 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d64994-c0b0-4c54-b496-4d9b1bc25650 service neutron] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:25:33.303720 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d64994-c0b0-4c54-b496-4d9b1bc25650 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:25:33.312514 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d64994-c0b0-4c54-b496-4d9b1bc25650 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:25:33.313966 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d64994-c0b0-4c54-b496-4d9b1bc25650 service neutron] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:25:33.323150 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1104/2183] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 12:25:33 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 12:25:36.609745 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29090700-dcc5-4afa-8ca7-ad0d1ab5cd14 None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:25:36.611232 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-29090700-dcc5-4afa-8ca7-ad0d1ab5cd14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:25:36.624060 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29090700-dcc5-4afa-8ca7-ad0d1ab5cd14 None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:25:36.629979 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5dfef4-5d33-4909-b093-aaf0690afb97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:25:36.630688 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5dfef4-5d33-4909-b093-aaf0690afb97 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:25:36.630971 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5dfef4-5d33-4909-b093-aaf0690afb97 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:25:36.631371 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5dfef4-5d33-4909-b093-aaf0690afb97 admin admin] PATH_INFO: `/v3/users/ee4eebde51f0441a8083e1b0c3c9eed2` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:25:36.635103 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5dfef4-5d33-4909-b093-aaf0690afb97 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ee4eebde51f0441a8083e1b0c3c9eed2)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:25:36.645212 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5dfef4-5d33-4909-b093-aaf0690afb97 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:25:36.717072 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-9e5dfef4-5d33-4909-b093-aaf0690afb97 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ee4eebde51f0441a8083e1b0c3c9eed2'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:25:36.747268 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-9e5dfef4-5d33-4909-b093-aaf0690afb97 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ee4eebde51f0441a8083e1b0c3c9eed2'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:25:36.758520 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-9e5dfef4-5d33-4909-b093-aaf0690afb97 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ee4eebde51f0441a8083e1b0c3c9eed2'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:25:36.760130 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-9e5dfef4-5d33-4909-b093-aaf0690afb97 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ee4eebde51f0441a8083e1b0c3c9eed2'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:25:36.771757 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-9e5dfef4-5d33-4909-b093-aaf0690afb97 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ee4eebde51f0441a8083e1b0c3c9eed2'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:25:36.779586 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1080/2184] 10.222.0.15 () {58 vars in 1224 bytes} [Thu Sep 15 12:25:36 2022] DELETE /identity/v3/users/ee4eebde51f0441a8083e1b0c3c9eed2 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:25:36.784976 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20fefaa-7911-4766-b12b-b36711fa061b None None] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:25:36.785504 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20fefaa-7911-4766-b12b-b36711fa061b None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:25:36.786184 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20fefaa-7911-4766-b12b-b36711fa061b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:25:36.813281 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.auth.core [None req-f20fefaa-7911-4766-b12b-b36711fa061b None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81449) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:25:36.816230 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-f20fefaa-7911-4766-b12b-b36711fa061b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:25:36.873054 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:25:36.918427 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1105/2185] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 12:25:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:25:36.928332 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5dfef4-5d33-4909-b093-aaf0690afb97 admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:25:36.929889 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-9e5dfef4-5d33-4909-b093-aaf0690afb97 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:25:36.946217 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:25:37.014725 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5dfef4-5d33-4909-b093-aaf0690afb97 admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:25:37.018386 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7130eeb6-577d-43c7-b158-84d3b9a6f286 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfbb4d6809064f9cb9e06626ed7d6814', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '540a32ef88764e8c8d9201ae22f6ddaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:25:37.019208 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7130eeb6-577d-43c7-b158-84d3b9a6f286 service neutron] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:25:37.019358 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7130eeb6-577d-43c7-b158-84d3b9a6f286 service neutron] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:25:37.019529 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7130eeb6-577d-43c7-b158-84d3b9a6f286 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:25:37.035038 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7130eeb6-577d-43c7-b158-84d3b9a6f286 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:25:37.037826 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7130eeb6-577d-43c7-b158-84d3b9a6f286 service neutron] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:25:37.051644 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1081/2186] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 12:25:36 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 12:25:37.383509 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20fefaa-7911-4766-b12b-b36711fa061b None None] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:25:37.385453 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-f20fefaa-7911-4766-b12b-b36711fa061b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:25:37.403660 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20fefaa-7911-4766-b12b-b36711fa061b None None] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:25:37.419219 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f37e52-faa9-4997-a3c4-65a7092be069 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:25:37.419897 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f37e52-faa9-4997-a3c4-65a7092be069 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:25:37.420195 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f37e52-faa9-4997-a3c4-65a7092be069 admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:25:37.420480 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f37e52-faa9-4997-a3c4-65a7092be069 admin admin] PATH_INFO: `/v3/projects/d82939b2da294a10bcd5e181fd7e9cb0` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:25:37.426656 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f37e52-faa9-4997-a3c4-65a7092be069 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d82939b2da294a10bcd5e181fd7e9cb0)` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:25:37.428648 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f37e52-faa9-4997-a3c4-65a7092be069 admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:25:37.536136 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-f0f37e52-faa9-4997-a3c4-65a7092be069 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd82939b2da294a10bcd5e181fd7e9cb0', 'request_id': 'req-f0f37e52-faa9-4997-a3c4-65a7092be069'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:25:37.548963 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-f0f37e52-faa9-4997-a3c4-65a7092be069 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd82939b2da294a10bcd5e181fd7e9cb0', 'request_id': 'req-f0f37e52-faa9-4997-a3c4-65a7092be069'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:25:37.562039 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-f0f37e52-faa9-4997-a3c4-65a7092be069 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd82939b2da294a10bcd5e181fd7e9cb0', 'request_id': 'req-f0f37e52-faa9-4997-a3c4-65a7092be069'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:25:37.581223 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-f0f37e52-faa9-4997-a3c4-65a7092be069 admin admin] The token cache is being invalidate because project d82939b2da294a10bcd5e181fd7e9cb0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81449) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 12:25:37.582608 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-f0f37e52-faa9-4997-a3c4-65a7092be069 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:25:37.583519 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-f0f37e52-faa9-4997-a3c4-65a7092be069 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:25:37.586620 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1106/2187] 10.222.0.15 () {58 vars in 1233 bytes} [Thu Sep 15 12:25:37 2022] DELETE /identity/v3/projects/d82939b2da294a10bcd5e181fd7e9cb0 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 12:25:37.590798 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7130eeb6-577d-43c7-b158-84d3b9a6f286 service neutron] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:25:37.609895 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7130eeb6-577d-43c7-b158-84d3b9a6f286 service neutron] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:25:37.622279 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6421e4f-9182-4994-8cb5-4e13f267afa5 None None] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:25:37.623412 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6421e4f-9182-4994-8cb5-4e13f267afa5 None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:25:37.623873 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6421e4f-9182-4994-8cb5-4e13f267afa5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:25:37.676600 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.auth.core [None req-e6421e4f-9182-4994-8cb5-4e13f267afa5 None None] MFA Rules not processed for user `20ca976222454c6c898d0e4b170b41fc`. Rule list: `[]` (Enabled: `True`). {{(pid=81449) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:25:37.680336 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-e6421e4f-9182-4994-8cb5-4e13f267afa5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:25:37.741527 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b0ff88-16e9-4234-8ff2-22bfcf7778f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:25:37.742390 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b0ff88-16e9-4234-8ff2-22bfcf7778f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:25:37.742703 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b0ff88-16e9-4234-8ff2-22bfcf7778f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:25:37.743316 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b0ff88-16e9-4234-8ff2-22bfcf7778f3 admin admin] PATH_INFO: `/v3/users/7d89d40d67814e6197b7ad21647df5d6` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:25:37.745370 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b0ff88-16e9-4234-8ff2-22bfcf7778f3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7d89d40d67814e6197b7ad21647df5d6)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:25:37.747901 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b0ff88-16e9-4234-8ff2-22bfcf7778f3 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:25:37.777085 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1107/2188] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 12:25:37 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 12:25:37.788100 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37d43c6-4ee7-41f0-96e5-fb4dfea218ad None None] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:25:37.788665 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37d43c6-4ee7-41f0-96e5-fb4dfea218ad None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:25:37.788665 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37d43c6-4ee7-41f0-96e5-fb4dfea218ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:25:37.819021 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.auth.core [None req-c37d43c6-4ee7-41f0-96e5-fb4dfea218ad None None] MFA Rules not processed for user `20ca976222454c6c898d0e4b170b41fc`. Rule list: `[]` (Enabled: `True`). {{(pid=81449) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:25:37.820394 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-c37d43c6-4ee7-41f0-96e5-fb4dfea218ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:25:37.832453 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1108/2189] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 12:25:37 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 12:25:37.844289 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-32b0ff88-16e9-4234-8ff2-22bfcf7778f3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7d89d40d67814e6197b7ad21647df5d6'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:25:37.847468 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37d43c6-4ee7-41f0-96e5-fb4dfea218ad None None] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:25:37.849181 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-c37d43c6-4ee7-41f0-96e5-fb4dfea218ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:25:37.861149 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-32b0ff88-16e9-4234-8ff2-22bfcf7778f3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7d89d40d67814e6197b7ad21647df5d6'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:25:37.871831 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-32b0ff88-16e9-4234-8ff2-22bfcf7778f3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7d89d40d67814e6197b7ad21647df5d6'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:25:37.872665 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-32b0ff88-16e9-4234-8ff2-22bfcf7778f3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7d89d40d67814e6197b7ad21647df5d6'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:25:37.882094 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-32b0ff88-16e9-4234-8ff2-22bfcf7778f3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7d89d40d67814e6197b7ad21647df5d6'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:25:37.891728 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1082/2190] 10.222.0.15 () {58 vars in 1224 bytes} [Thu Sep 15 12:25:37 2022] DELETE /identity/v3/users/7d89d40d67814e6197b7ad21647df5d6 => generated 0 bytes in 301 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:25:37.918274 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:25:37.947956 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37d43c6-4ee7-41f0-96e5-fb4dfea218ad None None] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:25:38.031614 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b58d75a-c229-4056-96ce-68a7d5707e47 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eface89e261840a2abb4206be26455c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '540a32ef88764e8c8d9201ae22f6ddaf', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:25:38.032378 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b58d75a-c229-4056-96ce-68a7d5707e47 service nova] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:25:38.032833 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b58d75a-c229-4056-96ce-68a7d5707e47 service nova] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:25:38.033162 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b58d75a-c229-4056-96ce-68a7d5707e47 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:25:38.036783 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-1b58d75a-c229-4056-96ce-68a7d5707e47 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:25:38.116500 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b58d75a-c229-4056-96ce-68a7d5707e47 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:25:38.118179 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b58d75a-c229-4056-96ce-68a7d5707e47 service nova] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:25:38.131665 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1109/2191] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 12:25:37 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 285 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 12:25:38.225541 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b0ff88-16e9-4234-8ff2-22bfcf7778f3 admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:25:38.227988 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-32b0ff88-16e9-4234-8ff2-22bfcf7778f3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:25:38.291731 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:25:38.342411 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b0ff88-16e9-4234-8ff2-22bfcf7778f3 admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:25:38.347540 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bde8b2c-8185-4be7-b9dd-82015d6850a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:25:38.348272 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bde8b2c-8185-4be7-b9dd-82015d6850a7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:25:38.348646 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bde8b2c-8185-4be7-b9dd-82015d6850a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:25:38.348913 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bde8b2c-8185-4be7-b9dd-82015d6850a7 admin admin] PATH_INFO: `/v3/projects/669bd636e9914855b21c1764cddd7038` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:25:38.350442 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bde8b2c-8185-4be7-b9dd-82015d6850a7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=669bd636e9914855b21c1764cddd7038)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:25:38.352947 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bde8b2c-8185-4be7-b9dd-82015d6850a7 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:25:38.418197 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-4bde8b2c-8185-4be7-b9dd-82015d6850a7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '669bd636e9914855b21c1764cddd7038', 'request_id': 'req-4bde8b2c-8185-4be7-b9dd-82015d6850a7'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:25:38.426968 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-4bde8b2c-8185-4be7-b9dd-82015d6850a7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '669bd636e9914855b21c1764cddd7038', 'request_id': 'req-4bde8b2c-8185-4be7-b9dd-82015d6850a7'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:25:38.433478 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-4bde8b2c-8185-4be7-b9dd-82015d6850a7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '669bd636e9914855b21c1764cddd7038', 'request_id': 'req-4bde8b2c-8185-4be7-b9dd-82015d6850a7'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:25:38.445395 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-4bde8b2c-8185-4be7-b9dd-82015d6850a7 admin admin] The token cache is being invalidate because project 669bd636e9914855b21c1764cddd7038 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81448) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 12:25:38.445737 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-4bde8b2c-8185-4be7-b9dd-82015d6850a7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:25:38.446350 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-4bde8b2c-8185-4be7-b9dd-82015d6850a7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:25:38.448588 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1083/2192] 10.222.0.15 () {58 vars in 1233 bytes} [Thu Sep 15 12:25:38 2022] DELETE /identity/v3/projects/669bd636e9914855b21c1764cddd7038 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:25:38.453549 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928a7430-9b33-4bc6-8b09-944a7c774480 None None] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:25:38.453879 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928a7430-9b33-4bc6-8b09-944a7c774480 None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:25:38.454160 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928a7430-9b33-4bc6-8b09-944a7c774480 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:25:38.481099 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.auth.core [None req-928a7430-9b33-4bc6-8b09-944a7c774480 None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81449) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:25:38.482477 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-928a7430-9b33-4bc6-8b09-944a7c774480 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:25:38.554734 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1110/2193] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 12:25:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:25:38.563240 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f207b1ba-e27f-41f8-8143-edceb631f3b7 None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:25:38.563526 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f207b1ba-e27f-41f8-8143-edceb631f3b7 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:25:38.563873 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f207b1ba-e27f-41f8-8143-edceb631f3b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:25:38.585741 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-f207b1ba-e27f-41f8-8143-edceb631f3b7 None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:25:38.586949 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-f207b1ba-e27f-41f8-8143-edceb631f3b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:25:38.595743 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1084/2194] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 12:25:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:25:38.600149 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928a7430-9b33-4bc6-8b09-944a7c774480 None None] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:25:38.610224 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928a7430-9b33-4bc6-8b09-944a7c774480 None None] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:25:38.615248 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03a72ab-a4b9-4ee3-83be-0bb2dc51dcf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:25:38.615804 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03a72ab-a4b9-4ee3-83be-0bb2dc51dcf0 admin admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:25:38.616086 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03a72ab-a4b9-4ee3-83be-0bb2dc51dcf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:25:38.616360 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03a72ab-a4b9-4ee3-83be-0bb2dc51dcf0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:25:38.617055 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03a72ab-a4b9-4ee3-83be-0bb2dc51dcf0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81449) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:25:38.617365 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03a72ab-a4b9-4ee3-83be-0bb2dc51dcf0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:25:38.619523 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03a72ab-a4b9-4ee3-83be-0bb2dc51dcf0 admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:25:38.636716 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1111/2195] 10.222.0.15 () {58 vars in 1153 bytes} [Thu Sep 15 12:25:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:25:38.641161 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f207b1ba-e27f-41f8-8143-edceb631f3b7 None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:25:38.650999 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f207b1ba-e27f-41f8-8143-edceb631f3b7 None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:25:38.657137 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3446f79e-2801-4064-8deb-30f45a543fe6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:25:38.658072 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3446f79e-2801-4064-8deb-30f45a543fe6 admin admin] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:25:38.658365 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3446f79e-2801-4064-8deb-30f45a543fe6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:25:38.658852 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3446f79e-2801-4064-8deb-30f45a543fe6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:25:38.659534 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3446f79e-2801-4064-8deb-30f45a543fe6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:25:38.661469 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3446f79e-2801-4064-8deb-30f45a543fe6 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:25:38.696826 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1085/2196] 10.222.0.15 () {60 vars in 1153 bytes} [Thu Sep 15 12:25:38 2022] POST /identity/v3/projects => generated 375 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 15 12:25:38.700483 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03a72ab-a4b9-4ee3-83be-0bb2dc51dcf0 admin admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:25:38.718195 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03a72ab-a4b9-4ee3-83be-0bb2dc51dcf0 admin admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:25:38.723652 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:25:38.782975 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc59b5ad-88cd-45e9-b598-7d0a0377b38b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:25:38.783665 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc59b5ad-88cd-45e9-b598-7d0a0377b38b admin admin] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:25:38.784008 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc59b5ad-88cd-45e9-b598-7d0a0377b38b admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:25:38.784351 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc59b5ad-88cd-45e9-b598-7d0a0377b38b admin admin] PATH_INFO: `/v3/users` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:25:38.785097 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc59b5ad-88cd-45e9-b598-7d0a0377b38b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:25:38.787085 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc59b5ad-88cd-45e9-b598-7d0a0377b38b admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:25:38.815178 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1112/2197] 10.222.0.15 () {60 vars in 1144 bytes} [Thu Sep 15 12:25:38 2022] POST /identity/v3/users => generated 288 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 12:25:38.819184 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3446f79e-2801-4064-8deb-30f45a543fe6 admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:25:38.835419 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3446f79e-2801-4064-8deb-30f45a543fe6 admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:25:38.840797 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf51f548-1ad6-4017-8cca-377425840334 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:25:38.841424 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf51f548-1ad6-4017-8cca-377425840334 admin admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:25:38.841699 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf51f548-1ad6-4017-8cca-377425840334 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:25:38.841971 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf51f548-1ad6-4017-8cca-377425840334 admin admin] PATH_INFO: `/v3/roles` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:25:38.842650 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf51f548-1ad6-4017-8cca-377425840334 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:25:38.844244 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf51f548-1ad6-4017-8cca-377425840334 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:25:38.855496 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1086/2198] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 12:25:38 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 12:25:38.859665 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc59b5ad-88cd-45e9-b598-7d0a0377b38b admin admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:25:38.870500 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc59b5ad-88cd-45e9-b598-7d0a0377b38b admin admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:25:38.878679 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3952c776-00b6-4fdb-bea9-7857172c2e46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:25:38.879301 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3952c776-00b6-4fdb-bea9-7857172c2e46 admin admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:25:38.879580 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3952c776-00b6-4fdb-bea9-7857172c2e46 admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:25:38.879853 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3952c776-00b6-4fdb-bea9-7857172c2e46 admin admin] PATH_INFO: `/v3/roles` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:25:38.880497 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3952c776-00b6-4fdb-bea9-7857172c2e46 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:25:38.882172 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3952c776-00b6-4fdb-bea9-7857172c2e46 admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:25:38.896277 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1113/2199] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 12:25:38 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 12:25:38.901068 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf51f548-1ad6-4017-8cca-377425840334 admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:25:38.911760 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf51f548-1ad6-4017-8cca-377425840334 admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:25:38.917336 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f83da08-1159-45ba-8009-78a73a21d95c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:25:38.918003 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f83da08-1159-45ba-8009-78a73a21d95c admin admin] REQUEST_METHOD: `PUT` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:25:38.918280 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f83da08-1159-45ba-8009-78a73a21d95c admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:25:38.918550 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f83da08-1159-45ba-8009-78a73a21d95c admin admin] PATH_INFO: `/v3/projects/059efc5c65a34cce800aa1df5e01a240/users/68b5601f167342d99d0f8de2c942acd1/roles/0679d6628a25438d90dc4bdc0401eda2` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:25:38.934389 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f83da08-1159-45ba-8009-78a73a21d95c admin admin] RBAC: Authorizing `identity:create_grant(project_id=059efc5c65a34cce800aa1df5e01a240, user_id=68b5601f167342d99d0f8de2c942acd1, role_id=0679d6628a25438d90dc4bdc0401eda2)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:25:38.937059 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f83da08-1159-45ba-8009-78a73a21d95c admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:25:38.949939 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1087/2200] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 15 12:25:38 2022] PUT /identity/v3/projects/059efc5c65a34cce800aa1df5e01a240/users/68b5601f167342d99d0f8de2c942acd1/roles/0679d6628a25438d90dc4bdc0401eda2 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:25:38.961463 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad9c19b-04f7-45e0-ae58-51460eccbdf0 None None] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:25:38.961746 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad9c19b-04f7-45e0-ae58-51460eccbdf0 None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:25:38.962016 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad9c19b-04f7-45e0-ae58-51460eccbdf0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:25:38.993132 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.auth.core [None req-9ad9c19b-04f7-45e0-ae58-51460eccbdf0 None None] MFA Rules not processed for user `68b5601f167342d99d0f8de2c942acd1`. Rule list: `[]` (Enabled: `True`). {{(pid=81449) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:25:38.994971 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-9ad9c19b-04f7-45e0-ae58-51460eccbdf0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:25:39.072397 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1114/2201] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 12:25:38 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:25:39.084266 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f83da08-1159-45ba-8009-78a73a21d95c admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:25:39.086035 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-6f83da08-1159-45ba-8009-78a73a21d95c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:25:39.145494 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f83da08-1159-45ba-8009-78a73a21d95c admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:25:39.149158 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e021e1-ae11-495e-8b14-5c322ce3baa1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8191d3ccb874529b32f8d0026db629d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '540a32ef88764e8c8d9201ae22f6ddaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:25:39.149804 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e021e1-ae11-495e-8b14-5c322ce3baa1 service cinder] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:25:39.150084 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e021e1-ae11-495e-8b14-5c322ce3baa1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:25:39.150357 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e021e1-ae11-495e-8b14-5c322ce3baa1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:25:39.158515 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e021e1-ae11-495e-8b14-5c322ce3baa1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:25:39.159930 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e021e1-ae11-495e-8b14-5c322ce3baa1 service cinder] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:25:39.169186 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1088/2202] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 12:25:39 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 12:25:42.370099 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad9c19b-04f7-45e0-ae58-51460eccbdf0 None None] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:25:42.433382 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad9c19b-04f7-45e0-ae58-51460eccbdf0 None None] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:25:42.437977 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39b785d-4ebb-4c9e-a3ff-011d822dac86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:25:42.438613 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39b785d-4ebb-4c9e-a3ff-011d822dac86 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:25:42.438889 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39b785d-4ebb-4c9e-a3ff-011d822dac86 admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:25:42.439178 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39b785d-4ebb-4c9e-a3ff-011d822dac86 admin admin] PATH_INFO: `/v3/users/68b5601f167342d99d0f8de2c942acd1` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:25:42.440309 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c39b785d-4ebb-4c9e-a3ff-011d822dac86 admin admin] RBAC: Authorizing `identity:delete_user(user_id=68b5601f167342d99d0f8de2c942acd1)` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:25:42.442255 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c39b785d-4ebb-4c9e-a3ff-011d822dac86 admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:25:42.463576 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=95,keystone:INSERT=3,keystone:DELETE=2 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:25:42.488885 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-c39b785d-4ebb-4c9e-a3ff-011d822dac86 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '68b5601f167342d99d0f8de2c942acd1'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:25:42.496541 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-c39b785d-4ebb-4c9e-a3ff-011d822dac86 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '68b5601f167342d99d0f8de2c942acd1'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:25:42.508430 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-c39b785d-4ebb-4c9e-a3ff-011d822dac86 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '68b5601f167342d99d0f8de2c942acd1'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:25:42.509673 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-c39b785d-4ebb-4c9e-a3ff-011d822dac86 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '68b5601f167342d99d0f8de2c942acd1'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:25:42.516814 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-c39b785d-4ebb-4c9e-a3ff-011d822dac86 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '68b5601f167342d99d0f8de2c942acd1'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:25:42.523588 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1115/2203] 10.222.0.15 () {58 vars in 1224 bytes} [Thu Sep 15 12:25:42 2022] DELETE /identity/v3/users/68b5601f167342d99d0f8de2c942acd1 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:25:42.528107 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52dbff1-c11b-4e56-8a71-434bde9e5daa None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:25:42.528555 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52dbff1-c11b-4e56-8a71-434bde9e5daa None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:25:42.528644 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52dbff1-c11b-4e56-8a71-434bde9e5daa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:25:42.535724 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:25:42.551455 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-b52dbff1-c11b-4e56-8a71-434bde9e5daa None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:25:42.552933 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-b52dbff1-c11b-4e56-8a71-434bde9e5daa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:25:42.614543 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1089/2204] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 12:25:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:25:42.625666 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39b785d-4ebb-4c9e-a3ff-011d822dac86 admin admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:25:42.627069 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-c39b785d-4ebb-4c9e-a3ff-011d822dac86 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:25:42.688184 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39b785d-4ebb-4c9e-a3ff-011d822dac86 admin admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:25:42.691850 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba55f1e9-5b74-432d-9eb8-57206cb3ef03 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfbb4d6809064f9cb9e06626ed7d6814', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '540a32ef88764e8c8d9201ae22f6ddaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:25:42.692473 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba55f1e9-5b74-432d-9eb8-57206cb3ef03 service neutron] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:25:42.692918 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba55f1e9-5b74-432d-9eb8-57206cb3ef03 service neutron] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:25:42.693198 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba55f1e9-5b74-432d-9eb8-57206cb3ef03 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:25:42.702717 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba55f1e9-5b74-432d-9eb8-57206cb3ef03 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:25:42.704162 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba55f1e9-5b74-432d-9eb8-57206cb3ef03 service neutron] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:25:42.713381 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1116/2205] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 12:25:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 12:25:42.986992 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52dbff1-c11b-4e56-8a71-434bde9e5daa None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:25:42.988444 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-b52dbff1-c11b-4e56-8a71-434bde9e5daa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:25:43.004921 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52dbff1-c11b-4e56-8a71-434bde9e5daa None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:25:43.010348 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7eaaaa1-d510-4626-851c-7f5719a75d36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:25:43.011105 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7eaaaa1-d510-4626-851c-7f5719a75d36 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:25:43.011289 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7eaaaa1-d510-4626-851c-7f5719a75d36 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:25:43.011563 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7eaaaa1-d510-4626-851c-7f5719a75d36 admin admin] PATH_INFO: `/v3/projects/059efc5c65a34cce800aa1df5e01a240` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:25:43.015612 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7eaaaa1-d510-4626-851c-7f5719a75d36 admin admin] RBAC: Authorizing `identity:delete_project(project_id=059efc5c65a34cce800aa1df5e01a240)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:25:43.017598 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7eaaaa1-d510-4626-851c-7f5719a75d36 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:25:43.064671 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-d7eaaaa1-d510-4626-851c-7f5719a75d36 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '059efc5c65a34cce800aa1df5e01a240', 'request_id': 'req-d7eaaaa1-d510-4626-851c-7f5719a75d36'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:25:43.072831 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-d7eaaaa1-d510-4626-851c-7f5719a75d36 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '059efc5c65a34cce800aa1df5e01a240', 'request_id': 'req-d7eaaaa1-d510-4626-851c-7f5719a75d36'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:25:43.078931 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-d7eaaaa1-d510-4626-851c-7f5719a75d36 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '059efc5c65a34cce800aa1df5e01a240', 'request_id': 'req-d7eaaaa1-d510-4626-851c-7f5719a75d36'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:25:43.091213 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-d7eaaaa1-d510-4626-851c-7f5719a75d36 admin admin] The token cache is being invalidate because project 059efc5c65a34cce800aa1df5e01a240 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81448) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 12:25:43.091503 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-d7eaaaa1-d510-4626-851c-7f5719a75d36 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:25:43.095240 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-d7eaaaa1-d510-4626-851c-7f5719a75d36 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:25:43.097230 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1090/2206] 10.222.0.15 () {58 vars in 1233 bytes} [Thu Sep 15 12:25:42 2022] DELETE /identity/v3/projects/059efc5c65a34cce800aa1df5e01a240 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:25:43.103652 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f999f6f2-bf32-488f-a96f-28573f1f16c4 None None] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:25:43.103945 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f999f6f2-bf32-488f-a96f-28573f1f16c4 None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:25:43.104216 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f999f6f2-bf32-488f-a96f-28573f1f16c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:25:43.124294 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.auth.core [None req-f999f6f2-bf32-488f-a96f-28573f1f16c4 None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81449) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:25:43.125460 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-f999f6f2-bf32-488f-a96f-28573f1f16c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:25:43.186546 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1117/2207] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 12:25:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:25:43.194473 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de50d535-2a15-4cc1-b340-f7908a72e685 None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:25:43.194767 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de50d535-2a15-4cc1-b340-f7908a72e685 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:25:43.195039 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de50d535-2a15-4cc1-b340-f7908a72e685 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:25:43.215917 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-de50d535-2a15-4cc1-b340-f7908a72e685 None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:25:43.217184 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-de50d535-2a15-4cc1-b340-f7908a72e685 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:25:43.225493 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1091/2208] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 12:25:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 12:25:43.230328 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f999f6f2-bf32-488f-a96f-28573f1f16c4 None None] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:25:43.240688 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f999f6f2-bf32-488f-a96f-28573f1f16c4 None None] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:25:43.245687 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af14ffed-1689-49a6-91e8-18efc545d799 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:25:43.246295 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af14ffed-1689-49a6-91e8-18efc545d799 admin admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:25:43.246575 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af14ffed-1689-49a6-91e8-18efc545d799 admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:25:43.246847 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af14ffed-1689-49a6-91e8-18efc545d799 admin admin] PATH_INFO: `/v3/domains` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:25:43.247499 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af14ffed-1689-49a6-91e8-18efc545d799 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81449) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:25:43.247800 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af14ffed-1689-49a6-91e8-18efc545d799 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:25:43.249449 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af14ffed-1689-49a6-91e8-18efc545d799 admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:25:43.264605 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1118/2209] 10.222.0.15 () {58 vars in 1153 bytes} [Thu Sep 15 12:25:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:25:43.265879 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de50d535-2a15-4cc1-b340-f7908a72e685 None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:25:43.275742 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de50d535-2a15-4cc1-b340-f7908a72e685 None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:25:43.280494 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c34c190-3970-43d0-9598-60e02da1ca5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:25:43.281146 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c34c190-3970-43d0-9598-60e02da1ca5c admin admin] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:25:43.281426 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c34c190-3970-43d0-9598-60e02da1ca5c admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:25:43.281778 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c34c190-3970-43d0-9598-60e02da1ca5c admin admin] PATH_INFO: `/v3/projects` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:25:43.282416 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c34c190-3970-43d0-9598-60e02da1ca5c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:25:43.284345 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c34c190-3970-43d0-9598-60e02da1ca5c admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:25:43.315424 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1092/2210] 10.222.0.15 () {60 vars in 1153 bytes} [Thu Sep 15 12:25:43 2022] POST /identity/v3/projects => generated 383 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 12:25:43.316407 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af14ffed-1689-49a6-91e8-18efc545d799 admin admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:25:43.326717 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af14ffed-1689-49a6-91e8-18efc545d799 admin admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:25:43.353659 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:DELETE=8,keystone:SELECT=91,keystone:INSERT=1 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:25:43.387416 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7a9463-833f-449e-a0ec-09b1f6c1ce92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:25:43.388050 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7a9463-833f-449e-a0ec-09b1f6c1ce92 admin admin] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:25:43.388264 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7a9463-833f-449e-a0ec-09b1f6c1ce92 admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:25:43.388468 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7a9463-833f-449e-a0ec-09b1f6c1ce92 admin admin] PATH_INFO: `/v3/users` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:25:43.389143 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7a9463-833f-449e-a0ec-09b1f6c1ce92 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:25:43.391108 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7a9463-833f-449e-a0ec-09b1f6c1ce92 admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:25:43.416744 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1119/2211] 10.222.0.15 () {60 vars in 1144 bytes} [Thu Sep 15 12:25:43 2022] POST /identity/v3/users => generated 292 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 15 12:25:43.417966 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c34c190-3970-43d0-9598-60e02da1ca5c admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:25:43.432704 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c34c190-3970-43d0-9598-60e02da1ca5c admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:25:43.437531 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224f5f6c-5e88-4d57-bd76-626deb55607c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:25:43.438146 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224f5f6c-5e88-4d57-bd76-626deb55607c admin admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:25:43.438472 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224f5f6c-5e88-4d57-bd76-626deb55607c admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:25:43.438717 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224f5f6c-5e88-4d57-bd76-626deb55607c admin admin] PATH_INFO: `/v3/roles` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:25:43.439363 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224f5f6c-5e88-4d57-bd76-626deb55607c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:25:43.441050 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-224f5f6c-5e88-4d57-bd76-626deb55607c admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:25:43.451832 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1093/2212] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 12:25:43 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 15 12:25:43.455834 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7a9463-833f-449e-a0ec-09b1f6c1ce92 admin admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:25:43.466504 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7a9463-833f-449e-a0ec-09b1f6c1ce92 admin admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:25:43.472019 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b9d008-693d-451d-9996-d5c19c25e454 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:25:43.472704 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b9d008-693d-451d-9996-d5c19c25e454 admin admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:25:43.473077 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b9d008-693d-451d-9996-d5c19c25e454 admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:25:43.473421 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b9d008-693d-451d-9996-d5c19c25e454 admin admin] PATH_INFO: `/v3/roles` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:25:43.474185 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b9d008-693d-451d-9996-d5c19c25e454 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:25:43.475851 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b9d008-693d-451d-9996-d5c19c25e454 admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:25:43.486847 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1120/2213] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 12:25:43 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 12:25:43.491095 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224f5f6c-5e88-4d57-bd76-626deb55607c admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:25:43.502349 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224f5f6c-5e88-4d57-bd76-626deb55607c admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:25:43.507013 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c803f2eb-71f9-4e1b-acb5-0efc1db156fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:25:43.507671 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c803f2eb-71f9-4e1b-acb5-0efc1db156fc admin admin] REQUEST_METHOD: `PUT` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:25:43.507952 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c803f2eb-71f9-4e1b-acb5-0efc1db156fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:25:43.508230 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c803f2eb-71f9-4e1b-acb5-0efc1db156fc admin admin] PATH_INFO: `/v3/projects/84505a7f2438403b8a94cbd7dbd8baa0/users/abc958be4a16424f92adab347fd9960d/roles/0679d6628a25438d90dc4bdc0401eda2` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:25:43.522941 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c803f2eb-71f9-4e1b-acb5-0efc1db156fc admin admin] RBAC: Authorizing `identity:create_grant(project_id=84505a7f2438403b8a94cbd7dbd8baa0, user_id=abc958be4a16424f92adab347fd9960d, role_id=0679d6628a25438d90dc4bdc0401eda2)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:25:43.526036 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c803f2eb-71f9-4e1b-acb5-0efc1db156fc admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:25:43.542854 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1094/2214] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 15 12:25:43 2022] PUT /identity/v3/projects/84505a7f2438403b8a94cbd7dbd8baa0/users/abc958be4a16424f92adab347fd9960d/roles/0679d6628a25438d90dc4bdc0401eda2 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:25:43.552879 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2bf16c-997e-403f-946d-90f1aa519fb3 None None] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:25:43.553166 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2bf16c-997e-403f-946d-90f1aa519fb3 None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:25:43.553445 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2bf16c-997e-403f-946d-90f1aa519fb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:25:43.594737 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.auth.core [None req-0e2bf16c-997e-403f-946d-90f1aa519fb3 None None] MFA Rules not processed for user `abc958be4a16424f92adab347fd9960d`. Rule list: `[]` (Enabled: `True`). {{(pid=81449) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:25:43.596342 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-0e2bf16c-997e-403f-946d-90f1aa519fb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:25:43.675605 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1121/2215] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 12:25:43 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:25:43.688153 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c803f2eb-71f9-4e1b-acb5-0efc1db156fc admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:25:43.689909 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-c803f2eb-71f9-4e1b-acb5-0efc1db156fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:25:43.719632 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:25:43.755325 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c803f2eb-71f9-4e1b-acb5-0efc1db156fc admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:25:43.759636 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32dba3c-354c-4a70-b04a-8c4c786165f0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8191d3ccb874529b32f8d0026db629d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '540a32ef88764e8c8d9201ae22f6ddaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:25:43.760430 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32dba3c-354c-4a70-b04a-8c4c786165f0 service cinder] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:25:43.760782 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32dba3c-354c-4a70-b04a-8c4c786165f0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:25:43.761155 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32dba3c-354c-4a70-b04a-8c4c786165f0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:25:43.773046 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a32dba3c-354c-4a70-b04a-8c4c786165f0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:25:43.775315 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a32dba3c-354c-4a70-b04a-8c4c786165f0 service cinder] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:25:43.786805 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1095/2216] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 12:25:43 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 12:25:43.820703 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01610ad-5eb0-4fed-a57b-7edfefe83432 None None] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:25:43.821026 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01610ad-5eb0-4fed-a57b-7edfefe83432 None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:25:43.821325 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01610ad-5eb0-4fed-a57b-7edfefe83432 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:25:43.847611 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.auth.core [None req-a01610ad-5eb0-4fed-a57b-7edfefe83432 None None] MFA Rules not processed for user `20ca976222454c6c898d0e4b170b41fc`. Rule list: `[]` (Enabled: `True`). {{(pid=81449) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:25:43.849278 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-a01610ad-5eb0-4fed-a57b-7edfefe83432 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:25:43.944344 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1122/2217] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 12:25:43 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:25:46.475362 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32dba3c-354c-4a70-b04a-8c4c786165f0 service cinder] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:25:46.545328 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32dba3c-354c-4a70-b04a-8c4c786165f0 service cinder] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:25:46.550841 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42da6a2e-0081-4000-ac1d-d010c680333f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:25:46.551623 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42da6a2e-0081-4000-ac1d-d010c680333f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:25:46.552005 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42da6a2e-0081-4000-ac1d-d010c680333f admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:25:46.552374 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42da6a2e-0081-4000-ac1d-d010c680333f admin admin] PATH_INFO: `/v3/users/abc958be4a16424f92adab347fd9960d` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:25:46.553916 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42da6a2e-0081-4000-ac1d-d010c680333f admin admin] RBAC: Authorizing `identity:delete_user(user_id=abc958be4a16424f92adab347fd9960d)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:25:46.555693 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42da6a2e-0081-4000-ac1d-d010c680333f admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:25:46.607553 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-42da6a2e-0081-4000-ac1d-d010c680333f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'abc958be4a16424f92adab347fd9960d'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:25:46.620273 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-42da6a2e-0081-4000-ac1d-d010c680333f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'abc958be4a16424f92adab347fd9960d'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:25:46.627447 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-42da6a2e-0081-4000-ac1d-d010c680333f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'abc958be4a16424f92adab347fd9960d'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:25:46.628052 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-42da6a2e-0081-4000-ac1d-d010c680333f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'abc958be4a16424f92adab347fd9960d'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:25:46.635685 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-42da6a2e-0081-4000-ac1d-d010c680333f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'abc958be4a16424f92adab347fd9960d'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:25:46.642566 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1096/2218] 10.222.0.15 () {58 vars in 1224 bytes} [Thu Sep 15 12:25:46 2022] DELETE /identity/v3/users/abc958be4a16424f92adab347fd9960d => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:25:46.647159 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345e6159-c7e1-4acf-a0f4-38a3609d2e84 None None] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:25:46.647767 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345e6159-c7e1-4acf-a0f4-38a3609d2e84 None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:25:46.648557 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345e6159-c7e1-4acf-a0f4-38a3609d2e84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:25:46.670654 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.auth.core [None req-345e6159-c7e1-4acf-a0f4-38a3609d2e84 None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81449) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:25:46.672002 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-345e6159-c7e1-4acf-a0f4-38a3609d2e84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:25:46.707073 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:25:46.742633 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1123/2219] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 12:25:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 12:25:46.753481 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42da6a2e-0081-4000-ac1d-d010c680333f admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:25:46.754884 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-42da6a2e-0081-4000-ac1d-d010c680333f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:25:46.807306 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:25:46.826151 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42da6a2e-0081-4000-ac1d-d010c680333f admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:25:46.830478 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d355361-f362-4c8f-a97f-f584405a9d90 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfbb4d6809064f9cb9e06626ed7d6814', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '540a32ef88764e8c8d9201ae22f6ddaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:25:46.831210 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d355361-f362-4c8f-a97f-f584405a9d90 service neutron] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:25:46.831587 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d355361-f362-4c8f-a97f-f584405a9d90 service neutron] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:25:46.831952 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d355361-f362-4c8f-a97f-f584405a9d90 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:25:46.844656 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d355361-f362-4c8f-a97f-f584405a9d90 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:25:46.846432 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d355361-f362-4c8f-a97f-f584405a9d90 service neutron] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:25:46.863479 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1097/2220] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 12:25:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 12:25:47.269236 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345e6159-c7e1-4acf-a0f4-38a3609d2e84 None None] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:25:47.270887 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-345e6159-c7e1-4acf-a0f4-38a3609d2e84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:25:47.294751 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345e6159-c7e1-4acf-a0f4-38a3609d2e84 None None] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:25:47.301281 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7638349c-b278-4b79-8c00-113546076e50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:25:47.303320 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7638349c-b278-4b79-8c00-113546076e50 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:25:47.303694 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7638349c-b278-4b79-8c00-113546076e50 admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:25:47.304042 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7638349c-b278-4b79-8c00-113546076e50 admin admin] PATH_INFO: `/v3/projects/84505a7f2438403b8a94cbd7dbd8baa0` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:25:47.305373 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7638349c-b278-4b79-8c00-113546076e50 admin admin] RBAC: Authorizing `identity:delete_project(project_id=84505a7f2438403b8a94cbd7dbd8baa0)` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:25:47.307877 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7638349c-b278-4b79-8c00-113546076e50 admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:25:47.374063 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-7638349c-b278-4b79-8c00-113546076e50 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '84505a7f2438403b8a94cbd7dbd8baa0', 'request_id': 'req-7638349c-b278-4b79-8c00-113546076e50'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:25:47.381804 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-7638349c-b278-4b79-8c00-113546076e50 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '84505a7f2438403b8a94cbd7dbd8baa0', 'request_id': 'req-7638349c-b278-4b79-8c00-113546076e50'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:25:47.388040 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-7638349c-b278-4b79-8c00-113546076e50 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '84505a7f2438403b8a94cbd7dbd8baa0', 'request_id': 'req-7638349c-b278-4b79-8c00-113546076e50'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:25:47.399622 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-7638349c-b278-4b79-8c00-113546076e50 admin admin] The token cache is being invalidate because project 84505a7f2438403b8a94cbd7dbd8baa0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81449) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 12:25:47.399847 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-7638349c-b278-4b79-8c00-113546076e50 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:25:47.400496 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-7638349c-b278-4b79-8c00-113546076e50 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:25:47.402342 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1124/2221] 10.222.0.15 () {58 vars in 1233 bytes} [Thu Sep 15 12:25:47 2022] DELETE /identity/v3/projects/84505a7f2438403b8a94cbd7dbd8baa0 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:25:56.852953 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=6 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:25:57.396405 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=36,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:26:00.593617 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d355361-f362-4c8f-a97f-f584405a9d90 service neutron] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:26:00.595939 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-4d355361-f362-4c8f-a97f-f584405a9d90 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:26:00.613048 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d355361-f362-4c8f-a97f-f584405a9d90 service neutron] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:26:00.690650 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf634ab9-c3f8-45f3-ac20-bc3d24ebaa75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:26:00.691404 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf634ab9-c3f8-45f3-ac20-bc3d24ebaa75 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:26:00.691740 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf634ab9-c3f8-45f3-ac20-bc3d24ebaa75 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:26:00.692019 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf634ab9-c3f8-45f3-ac20-bc3d24ebaa75 admin admin] PATH_INFO: `/v3/users/9699fd2d69f1499ea38a861085bd1821` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:26:00.693628 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf634ab9-c3f8-45f3-ac20-bc3d24ebaa75 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9699fd2d69f1499ea38a861085bd1821)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:26:00.695674 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf634ab9-c3f8-45f3-ac20-bc3d24ebaa75 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:26:00.771963 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-bf634ab9-c3f8-45f3-ac20-bc3d24ebaa75 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9699fd2d69f1499ea38a861085bd1821'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:26:00.787725 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-bf634ab9-c3f8-45f3-ac20-bc3d24ebaa75 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9699fd2d69f1499ea38a861085bd1821'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:26:00.796625 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-bf634ab9-c3f8-45f3-ac20-bc3d24ebaa75 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9699fd2d69f1499ea38a861085bd1821'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:26:00.797354 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-bf634ab9-c3f8-45f3-ac20-bc3d24ebaa75 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9699fd2d69f1499ea38a861085bd1821'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:26:00.804719 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-bf634ab9-c3f8-45f3-ac20-bc3d24ebaa75 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9699fd2d69f1499ea38a861085bd1821'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:26:00.811335 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1098/2222] 10.222.0.15 () {58 vars in 1224 bytes} [Thu Sep 15 12:26:00 2022] DELETE /identity/v3/users/9699fd2d69f1499ea38a861085bd1821 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:26:00.815950 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc773c6-f5d8-41b3-a69a-f244b53422a5 None None] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:26:00.816393 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc773c6-f5d8-41b3-a69a-f244b53422a5 None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:26:00.816586 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc773c6-f5d8-41b3-a69a-f244b53422a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:26:00.837643 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.auth.core [None req-2dc773c6-f5d8-41b3-a69a-f244b53422a5 None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81449) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:26:00.838888 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-2dc773c6-f5d8-41b3-a69a-f244b53422a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:26:00.905794 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1125/2223] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 12:26:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 12:26:00.916682 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf634ab9-c3f8-45f3-ac20-bc3d24ebaa75 admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:26:00.918102 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-bf634ab9-c3f8-45f3-ac20-bc3d24ebaa75 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:26:00.994679 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf634ab9-c3f8-45f3-ac20-bc3d24ebaa75 admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:26:00.998410 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771f9a47-cb94-4969-b397-946a366a597e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfbb4d6809064f9cb9e06626ed7d6814', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '540a32ef88764e8c8d9201ae22f6ddaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:26:00.999022 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771f9a47-cb94-4969-b397-946a366a597e service neutron] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:26:00.999303 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771f9a47-cb94-4969-b397-946a366a597e service neutron] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:26:00.999574 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771f9a47-cb94-4969-b397-946a366a597e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:26:01.009051 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771f9a47-cb94-4969-b397-946a366a597e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:26:01.010538 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771f9a47-cb94-4969-b397-946a366a597e service neutron] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:26:01.020418 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1099/2224] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 12:26:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 12:26:01.127931 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc773c6-f5d8-41b3-a69a-f244b53422a5 None None] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:26:01.132965 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-2dc773c6-f5d8-41b3-a69a-f244b53422a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:26:01.157510 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc773c6-f5d8-41b3-a69a-f244b53422a5 None None] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:26:01.165004 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92fedd29-4fc7-4079-a1bb-9e844595dd60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:26:01.165734 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fedd29-4fc7-4079-a1bb-9e844595dd60 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:26:01.167333 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fedd29-4fc7-4079-a1bb-9e844595dd60 admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:26:01.167610 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fedd29-4fc7-4079-a1bb-9e844595dd60 admin admin] PATH_INFO: `/v3/users/ed5e8c2d770a42848f22ed2243f2a5b0` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:26:01.169199 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92fedd29-4fc7-4079-a1bb-9e844595dd60 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ed5e8c2d770a42848f22ed2243f2a5b0)` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:26:01.180504 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92fedd29-4fc7-4079-a1bb-9e844595dd60 admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:26:01.233267 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-92fedd29-4fc7-4079-a1bb-9e844595dd60 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ed5e8c2d770a42848f22ed2243f2a5b0'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:26:01.243025 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-92fedd29-4fc7-4079-a1bb-9e844595dd60 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ed5e8c2d770a42848f22ed2243f2a5b0'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:26:01.259773 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-92fedd29-4fc7-4079-a1bb-9e844595dd60 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ed5e8c2d770a42848f22ed2243f2a5b0'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:26:01.260497 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-92fedd29-4fc7-4079-a1bb-9e844595dd60 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ed5e8c2d770a42848f22ed2243f2a5b0'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:26:01.268244 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-92fedd29-4fc7-4079-a1bb-9e844595dd60 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ed5e8c2d770a42848f22ed2243f2a5b0'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:26:01.274979 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1126/2225] 10.222.0.15 () {58 vars in 1224 bytes} [Thu Sep 15 12:26:01 2022] DELETE /identity/v3/users/ed5e8c2d770a42848f22ed2243f2a5b0 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:26:01.280096 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f994d6-b0d8-4f55-bf5b-e117167b9e3c None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:26:01.280456 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f994d6-b0d8-4f55-bf5b-e117167b9e3c None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:26:01.280726 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f994d6-b0d8-4f55-bf5b-e117167b9e3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:26:01.303826 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-19f994d6-b0d8-4f55-bf5b-e117167b9e3c None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:26:01.305322 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-19f994d6-b0d8-4f55-bf5b-e117167b9e3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:26:01.325289 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:26:01.377875 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1100/2226] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 12:26:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:26:01.391445 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92fedd29-4fc7-4079-a1bb-9e844595dd60 admin admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:26:01.393179 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-92fedd29-4fc7-4079-a1bb-9e844595dd60 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:26:01.401231 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f994d6-b0d8-4f55-bf5b-e117167b9e3c None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:26:01.403146 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-19f994d6-b0d8-4f55-bf5b-e117167b9e3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:26:01.421817 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f994d6-b0d8-4f55-bf5b-e117167b9e3c None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:26:01.427927 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f874b6a-9c09-44f3-9c2e-35d61ff5bb70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:26:01.428879 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f874b6a-9c09-44f3-9c2e-35d61ff5bb70 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:26:01.429165 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f874b6a-9c09-44f3-9c2e-35d61ff5bb70 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:26:01.430075 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f874b6a-9c09-44f3-9c2e-35d61ff5bb70 admin admin] PATH_INFO: `/v3/projects/1833fa8600d84bcb892dfcab47e03c17` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:26:01.431392 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f874b6a-9c09-44f3-9c2e-35d61ff5bb70 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1833fa8600d84bcb892dfcab47e03c17)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:26:01.433804 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f874b6a-9c09-44f3-9c2e-35d61ff5bb70 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:26:01.460863 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92fedd29-4fc7-4079-a1bb-9e844595dd60 admin admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:26:01.466468 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684d5725-f704-42f0-b11d-86accd7b3ad2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfbb4d6809064f9cb9e06626ed7d6814', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '540a32ef88764e8c8d9201ae22f6ddaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:26:01.467411 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684d5725-f704-42f0-b11d-86accd7b3ad2 service neutron] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:26:01.467699 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684d5725-f704-42f0-b11d-86accd7b3ad2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:26:01.467967 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-684d5725-f704-42f0-b11d-86accd7b3ad2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:26:01.479666 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684d5725-f704-42f0-b11d-86accd7b3ad2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:26:01.482190 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-684d5725-f704-42f0-b11d-86accd7b3ad2 service neutron] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:26:01.495055 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1127/2227] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 12:26:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 12:26:01.498906 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-1f874b6a-9c09-44f3-9c2e-35d61ff5bb70 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1833fa8600d84bcb892dfcab47e03c17', 'request_id': 'req-1f874b6a-9c09-44f3-9c2e-35d61ff5bb70'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:26:01.508117 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-1f874b6a-9c09-44f3-9c2e-35d61ff5bb70 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1833fa8600d84bcb892dfcab47e03c17', 'request_id': 'req-1f874b6a-9c09-44f3-9c2e-35d61ff5bb70'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:26:01.523962 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-1f874b6a-9c09-44f3-9c2e-35d61ff5bb70 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1833fa8600d84bcb892dfcab47e03c17', 'request_id': 'req-1f874b6a-9c09-44f3-9c2e-35d61ff5bb70'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:26:01.536451 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-1f874b6a-9c09-44f3-9c2e-35d61ff5bb70 admin admin] The token cache is being invalidate because project 1833fa8600d84bcb892dfcab47e03c17 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81448) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 12:26:01.536974 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-1f874b6a-9c09-44f3-9c2e-35d61ff5bb70 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:26:01.537667 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-1f874b6a-9c09-44f3-9c2e-35d61ff5bb70 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:26:01.539207 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1101/2228] 10.222.0.15 () {58 vars in 1233 bytes} [Thu Sep 15 12:26:01 2022] DELETE /identity/v3/projects/1833fa8600d84bcb892dfcab47e03c17 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 12:26:01.983861 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684d5725-f704-42f0-b11d-86accd7b3ad2 service neutron] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:26:01.985395 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-684d5725-f704-42f0-b11d-86accd7b3ad2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:26:01.999921 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-684d5725-f704-42f0-b11d-86accd7b3ad2 service neutron] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:26:02.013009 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:26:02.062013 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf89629-3726-4187-8359-16d04d0b22a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:26:02.062556 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf89629-3726-4187-8359-16d04d0b22a7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:26:02.062835 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf89629-3726-4187-8359-16d04d0b22a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:26:02.063107 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf89629-3726-4187-8359-16d04d0b22a7 admin admin] PATH_INFO: `/v3/projects/68f00d468c394abcb256b3904d8ad3bf` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:26:02.064224 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf89629-3726-4187-8359-16d04d0b22a7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=68f00d468c394abcb256b3904d8ad3bf)` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:26:02.066157 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf89629-3726-4187-8359-16d04d0b22a7 admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:26:02.111269 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-9bf89629-3726-4187-8359-16d04d0b22a7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '68f00d468c394abcb256b3904d8ad3bf', 'request_id': 'req-9bf89629-3726-4187-8359-16d04d0b22a7'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:26:02.118719 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-9bf89629-3726-4187-8359-16d04d0b22a7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '68f00d468c394abcb256b3904d8ad3bf', 'request_id': 'req-9bf89629-3726-4187-8359-16d04d0b22a7'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:26:02.125094 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-9bf89629-3726-4187-8359-16d04d0b22a7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '68f00d468c394abcb256b3904d8ad3bf', 'request_id': 'req-9bf89629-3726-4187-8359-16d04d0b22a7'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:26:02.135917 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-9bf89629-3726-4187-8359-16d04d0b22a7 admin admin] The token cache is being invalidate because project 68f00d468c394abcb256b3904d8ad3bf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81449) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 12:26:02.136204 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-9bf89629-3726-4187-8359-16d04d0b22a7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:26:02.136884 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-9bf89629-3726-4187-8359-16d04d0b22a7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:26:02.138353 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1128/2229] 10.222.0.15 () {58 vars in 1233 bytes} [Thu Sep 15 12:26:01 2022] DELETE /identity/v3/projects/68f00d468c394abcb256b3904d8ad3bf => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:26:02.143289 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecf1377-e2cd-4966-9eac-5bebad7ba184 None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:26:02.143761 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecf1377-e2cd-4966-9eac-5bebad7ba184 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:26:02.143857 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecf1377-e2cd-4966-9eac-5bebad7ba184 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:26:02.162997 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-4ecf1377-e2cd-4966-9eac-5bebad7ba184 None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:26:02.164172 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-4ecf1377-e2cd-4966-9eac-5bebad7ba184 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:26:02.224621 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1102/2230] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 12:26:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 12:26:02.233049 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9453081c-42cb-4738-9562-031856495a15 None None] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:26:02.233345 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9453081c-42cb-4738-9562-031856495a15 None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:26:02.233650 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9453081c-42cb-4738-9562-031856495a15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:26:02.254638 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.auth.core [None req-9453081c-42cb-4738-9562-031856495a15 None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81449) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:26:02.255850 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-9453081c-42cb-4738-9562-031856495a15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:26:02.263741 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1129/2231] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 12:26:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:26:02.267910 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecf1377-e2cd-4966-9eac-5bebad7ba184 None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:26:02.278039 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecf1377-e2cd-4966-9eac-5bebad7ba184 None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:26:02.282527 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e376ba2c-d446-46e2-8456-e1ce5d9204b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:26:02.283133 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e376ba2c-d446-46e2-8456-e1ce5d9204b6 admin admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:26:02.283419 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e376ba2c-d446-46e2-8456-e1ce5d9204b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:26:02.283685 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e376ba2c-d446-46e2-8456-e1ce5d9204b6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:26:02.284689 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e376ba2c-d446-46e2-8456-e1ce5d9204b6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:26:02.284689 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e376ba2c-d446-46e2-8456-e1ce5d9204b6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:26:02.286298 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e376ba2c-d446-46e2-8456-e1ce5d9204b6 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:26:02.298610 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1103/2232] 10.222.0.15 () {58 vars in 1153 bytes} [Thu Sep 15 12:26:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:26:02.302684 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9453081c-42cb-4738-9562-031856495a15 None None] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:26:02.313094 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9453081c-42cb-4738-9562-031856495a15 None None] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:26:02.317754 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064036b0-9a1a-40aa-9216-4156c8e161c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:26:02.318355 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064036b0-9a1a-40aa-9216-4156c8e161c1 admin admin] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:26:02.318656 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064036b0-9a1a-40aa-9216-4156c8e161c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:26:02.318903 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064036b0-9a1a-40aa-9216-4156c8e161c1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:26:02.319526 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-064036b0-9a1a-40aa-9216-4156c8e161c1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:26:02.321472 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-064036b0-9a1a-40aa-9216-4156c8e161c1 admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:26:02.349739 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1130/2233] 10.222.0.15 () {60 vars in 1153 bytes} [Thu Sep 15 12:26:02 2022] POST /identity/v3/projects => generated 361 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 12:26:02.354131 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e376ba2c-d446-46e2-8456-e1ce5d9204b6 admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:26:02.364116 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e376ba2c-d446-46e2-8456-e1ce5d9204b6 admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:26:02.396464 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:26:02.425089 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c9d8b3-4983-40a2-b502-30d8931f9bc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:26:02.425699 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c9d8b3-4983-40a2-b502-30d8931f9bc4 admin admin] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:26:02.425977 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c9d8b3-4983-40a2-b502-30d8931f9bc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:26:02.426252 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c9d8b3-4983-40a2-b502-30d8931f9bc4 admin admin] PATH_INFO: `/v3/users` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:26:02.426869 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c9d8b3-4983-40a2-b502-30d8931f9bc4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:26:02.428784 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c9d8b3-4983-40a2-b502-30d8931f9bc4 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:26:02.449262 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1104/2234] 10.222.0.15 () {60 vars in 1144 bytes} [Thu Sep 15 12:26:02 2022] POST /identity/v3/users => generated 281 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 12:26:02.453304 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064036b0-9a1a-40aa-9216-4156c8e161c1 admin admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:26:02.463920 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064036b0-9a1a-40aa-9216-4156c8e161c1 admin admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:26:02.468436 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f3e951-b285-483e-ae16-f9fc3e266024 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:26:02.469327 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f3e951-b285-483e-ae16-f9fc3e266024 admin admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:26:02.469602 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f3e951-b285-483e-ae16-f9fc3e266024 admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:26:02.469868 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f3e951-b285-483e-ae16-f9fc3e266024 admin admin] PATH_INFO: `/v3/roles` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:26:02.470504 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f3e951-b285-483e-ae16-f9fc3e266024 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:26:02.472075 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76f3e951-b285-483e-ae16-f9fc3e266024 admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:26:02.481653 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1131/2235] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 12:26:02 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 12:26:02.485363 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c9d8b3-4983-40a2-b502-30d8931f9bc4 admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:26:02.495809 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c9d8b3-4983-40a2-b502-30d8931f9bc4 admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:26:02.500524 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95fc9553-c01b-4626-8dbb-4590702add63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:26:02.501146 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fc9553-c01b-4626-8dbb-4590702add63 admin admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:26:02.501420 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fc9553-c01b-4626-8dbb-4590702add63 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:26:02.501686 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fc9553-c01b-4626-8dbb-4590702add63 admin admin] PATH_INFO: `/v3/roles` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:26:02.502306 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95fc9553-c01b-4626-8dbb-4590702add63 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:26:02.503862 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95fc9553-c01b-4626-8dbb-4590702add63 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:26:02.514620 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1105/2236] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 12:26:02 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 12:26:02.517851 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f3e951-b285-483e-ae16-f9fc3e266024 admin admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:26:02.527660 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f3e951-b285-483e-ae16-f9fc3e266024 admin admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:26:02.532436 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd3086a-0b8f-46b6-a20d-a1ec90ad08ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:26:02.533137 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd3086a-0b8f-46b6-a20d-a1ec90ad08ce admin admin] REQUEST_METHOD: `PUT` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:26:02.533415 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd3086a-0b8f-46b6-a20d-a1ec90ad08ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:26:02.533688 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd3086a-0b8f-46b6-a20d-a1ec90ad08ce admin admin] PATH_INFO: `/v3/projects/cb5604dc627247ada64d6d4bf74f2f38/users/e0b07b7502c74a49a0e1431b798ab774/roles/0679d6628a25438d90dc4bdc0401eda2` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:26:02.548128 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd3086a-0b8f-46b6-a20d-a1ec90ad08ce admin admin] RBAC: Authorizing `identity:create_grant(project_id=cb5604dc627247ada64d6d4bf74f2f38, user_id=e0b07b7502c74a49a0e1431b798ab774, role_id=0679d6628a25438d90dc4bdc0401eda2)` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:26:02.550830 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd3086a-0b8f-46b6-a20d-a1ec90ad08ce admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:26:02.565026 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1132/2237] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 15 12:26:02 2022] PUT /identity/v3/projects/cb5604dc627247ada64d6d4bf74f2f38/users/e0b07b7502c74a49a0e1431b798ab774/roles/0679d6628a25438d90dc4bdc0401eda2 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 12:26:02.575694 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11043ffd-8211-4e91-9c94-1586c6653e5c None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:26:02.575984 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11043ffd-8211-4e91-9c94-1586c6653e5c None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:26:02.576259 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11043ffd-8211-4e91-9c94-1586c6653e5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:26:02.604371 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-11043ffd-8211-4e91-9c94-1586c6653e5c None None] MFA Rules not processed for user `e0b07b7502c74a49a0e1431b798ab774`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:26:02.605674 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-11043ffd-8211-4e91-9c94-1586c6653e5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:26:02.674316 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1106/2238] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 12:26:02 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:26:02.708887 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd3086a-0b8f-46b6-a20d-a1ec90ad08ce admin admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:26:02.710596 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-3cd3086a-0b8f-46b6-a20d-a1ec90ad08ce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:26:02.755297 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:26:02.768675 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd3086a-0b8f-46b6-a20d-a1ec90ad08ce admin admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:26:02.772295 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efec71dc-bcf7-4415-87a6-9f6473a8fa9b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8191d3ccb874529b32f8d0026db629d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '540a32ef88764e8c8d9201ae22f6ddaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:26:02.772926 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efec71dc-bcf7-4415-87a6-9f6473a8fa9b service cinder] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:26:02.773206 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efec71dc-bcf7-4415-87a6-9f6473a8fa9b service cinder] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:26:02.773477 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efec71dc-bcf7-4415-87a6-9f6473a8fa9b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:26:02.781628 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efec71dc-bcf7-4415-87a6-9f6473a8fa9b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:26:02.783047 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efec71dc-bcf7-4415-87a6-9f6473a8fa9b service cinder] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:26:02.791588 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1133/2239] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 12:26:02 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 12:26:02.807075 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11043ffd-8211-4e91-9c94-1586c6653e5c None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:26:02.870313 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11043ffd-8211-4e91-9c94-1586c6653e5c None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:26:02.874812 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b411e6-4549-4c18-9935-73d872239790 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:26:02.875442 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b411e6-4549-4c18-9935-73d872239790 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:26:02.875719 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b411e6-4549-4c18-9935-73d872239790 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:26:02.875992 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b411e6-4549-4c18-9935-73d872239790 admin admin] PATH_INFO: `/v3/users/e0b07b7502c74a49a0e1431b798ab774` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:26:02.877302 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b411e6-4549-4c18-9935-73d872239790 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e0b07b7502c74a49a0e1431b798ab774)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:26:02.879097 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b411e6-4549-4c18-9935-73d872239790 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:26:02.921488 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=91,keystone:INSERT=3,keystone:DELETE=6 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:26:02.924490 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-b4b411e6-4549-4c18-9935-73d872239790 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e0b07b7502c74a49a0e1431b798ab774'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:26:02.937330 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-b4b411e6-4549-4c18-9935-73d872239790 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e0b07b7502c74a49a0e1431b798ab774'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:26:02.945115 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-b4b411e6-4549-4c18-9935-73d872239790 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e0b07b7502c74a49a0e1431b798ab774'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:26:02.945753 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-b4b411e6-4549-4c18-9935-73d872239790 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e0b07b7502c74a49a0e1431b798ab774'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:26:02.953288 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-b4b411e6-4549-4c18-9935-73d872239790 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e0b07b7502c74a49a0e1431b798ab774'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:26:02.963955 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1107/2240] 10.222.0.15 () {58 vars in 1224 bytes} [Thu Sep 15 12:26:02 2022] DELETE /identity/v3/users/e0b07b7502c74a49a0e1431b798ab774 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:26:02.968408 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e218e0e6-54e3-47fd-929e-57a288b87a50 None None] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:26:02.968712 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e218e0e6-54e3-47fd-929e-57a288b87a50 None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:26:02.969305 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e218e0e6-54e3-47fd-929e-57a288b87a50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:26:02.988533 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.auth.core [None req-e218e0e6-54e3-47fd-929e-57a288b87a50 None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81449) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:26:02.989765 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-e218e0e6-54e3-47fd-929e-57a288b87a50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:26:03.048461 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1134/2241] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 12:26:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:26:03.058128 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b411e6-4549-4c18-9935-73d872239790 admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:26:03.059505 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-b4b411e6-4549-4c18-9935-73d872239790 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:26:03.115444 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b411e6-4549-4c18-9935-73d872239790 admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:26:03.118977 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4078097-9352-4644-988f-023f38552e34 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfbb4d6809064f9cb9e06626ed7d6814', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '540a32ef88764e8c8d9201ae22f6ddaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:26:03.119570 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4078097-9352-4644-988f-023f38552e34 service neutron] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:26:03.119846 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4078097-9352-4644-988f-023f38552e34 service neutron] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:26:03.120117 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4078097-9352-4644-988f-023f38552e34 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:26:03.128818 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4078097-9352-4644-988f-023f38552e34 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:26:03.130212 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4078097-9352-4644-988f-023f38552e34 service neutron] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:26:03.140146 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1108/2242] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 12:26:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 12:26:03.400891 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e218e0e6-54e3-47fd-929e-57a288b87a50 None None] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:26:03.402023 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-e218e0e6-54e3-47fd-929e-57a288b87a50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:26:03.415796 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e218e0e6-54e3-47fd-929e-57a288b87a50 None None] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:26:03.420990 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66cbbff2-7e34-47eb-ace8-ee4a2afe1b13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:26:03.421724 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cbbff2-7e34-47eb-ace8-ee4a2afe1b13 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:26:03.422126 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cbbff2-7e34-47eb-ace8-ee4a2afe1b13 admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:26:03.422563 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cbbff2-7e34-47eb-ace8-ee4a2afe1b13 admin admin] PATH_INFO: `/v3/projects/cb5604dc627247ada64d6d4bf74f2f38` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:26:03.423794 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66cbbff2-7e34-47eb-ace8-ee4a2afe1b13 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cb5604dc627247ada64d6d4bf74f2f38)` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:26:03.426171 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66cbbff2-7e34-47eb-ace8-ee4a2afe1b13 admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:26:03.476669 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-66cbbff2-7e34-47eb-ace8-ee4a2afe1b13 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cb5604dc627247ada64d6d4bf74f2f38', 'request_id': 'req-66cbbff2-7e34-47eb-ace8-ee4a2afe1b13'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:26:03.484692 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-66cbbff2-7e34-47eb-ace8-ee4a2afe1b13 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cb5604dc627247ada64d6d4bf74f2f38', 'request_id': 'req-66cbbff2-7e34-47eb-ace8-ee4a2afe1b13'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:26:03.493492 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-66cbbff2-7e34-47eb-ace8-ee4a2afe1b13 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cb5604dc627247ada64d6d4bf74f2f38', 'request_id': 'req-66cbbff2-7e34-47eb-ace8-ee4a2afe1b13'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:26:03.505687 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-66cbbff2-7e34-47eb-ace8-ee4a2afe1b13 admin admin] The token cache is being invalidate because project cb5604dc627247ada64d6d4bf74f2f38 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81449) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 12:26:03.505978 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-66cbbff2-7e34-47eb-ace8-ee4a2afe1b13 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:26:03.506633 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-66cbbff2-7e34-47eb-ace8-ee4a2afe1b13 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:26:03.508296 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1135/2243] 10.222.0.15 () {58 vars in 1233 bytes} [Thu Sep 15 12:26:03 2022] DELETE /identity/v3/projects/cb5604dc627247ada64d6d4bf74f2f38 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 12:26:03.513526 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810ade5c-5790-4a1f-9c45-5e09cd52c710 None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:26:03.514023 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810ade5c-5790-4a1f-9c45-5e09cd52c710 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:26:03.514126 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810ade5c-5790-4a1f-9c45-5e09cd52c710 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:26:03.534187 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-810ade5c-5790-4a1f-9c45-5e09cd52c710 None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:26:03.535422 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-810ade5c-5790-4a1f-9c45-5e09cd52c710 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:26:03.596223 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1109/2244] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 12:26:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:26:03.605549 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145b8bc8-8873-4981-b257-e58a687699da None None] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:26:03.605841 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145b8bc8-8873-4981-b257-e58a687699da None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:26:03.606116 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145b8bc8-8873-4981-b257-e58a687699da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:26:03.627071 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.auth.core [None req-145b8bc8-8873-4981-b257-e58a687699da None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81449) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:26:03.628299 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-145b8bc8-8873-4981-b257-e58a687699da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:26:03.636064 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1136/2245] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 12:26:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:26:03.640297 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810ade5c-5790-4a1f-9c45-5e09cd52c710 None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:26:03.649761 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810ade5c-5790-4a1f-9c45-5e09cd52c710 None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:26:03.654544 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75934dff-a6e9-4cdb-93cf-ba2c8b0ead6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:26:03.655144 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75934dff-a6e9-4cdb-93cf-ba2c8b0ead6a admin admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:26:03.655416 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75934dff-a6e9-4cdb-93cf-ba2c8b0ead6a admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:26:03.655741 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75934dff-a6e9-4cdb-93cf-ba2c8b0ead6a admin admin] PATH_INFO: `/v3/domains` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:26:03.656331 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75934dff-a6e9-4cdb-93cf-ba2c8b0ead6a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:26:03.656699 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75934dff-a6e9-4cdb-93cf-ba2c8b0ead6a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:26:03.658271 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75934dff-a6e9-4cdb-93cf-ba2c8b0ead6a admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:26:03.670293 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1110/2246] 10.222.0.15 () {58 vars in 1153 bytes} [Thu Sep 15 12:26:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:26:03.674470 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145b8bc8-8873-4981-b257-e58a687699da None None] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:26:03.684158 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145b8bc8-8873-4981-b257-e58a687699da None None] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:26:03.688672 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c433c8b0-fbb5-439e-adc0-79e957414438 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:26:03.689278 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c433c8b0-fbb5-439e-adc0-79e957414438 admin admin] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:26:03.689552 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c433c8b0-fbb5-439e-adc0-79e957414438 admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:26:03.689821 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c433c8b0-fbb5-439e-adc0-79e957414438 admin admin] PATH_INFO: `/v3/projects` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:26:03.690443 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c433c8b0-fbb5-439e-adc0-79e957414438 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:26:03.692359 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c433c8b0-fbb5-439e-adc0-79e957414438 admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:26:03.719181 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1137/2247] 10.222.0.15 () {60 vars in 1153 bytes} [Thu Sep 15 12:26:03 2022] POST /identity/v3/projects => generated 377 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 15 12:26:03.725776 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75934dff-a6e9-4cdb-93cf-ba2c8b0ead6a admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:26:03.736821 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75934dff-a6e9-4cdb-93cf-ba2c8b0ead6a admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:26:03.783888 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:DELETE=4,keystone:SELECT=95,keystone:INSERT=1 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:26:03.796057 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c21525-52dc-4046-92b7-191abb2e8050 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:26:03.797033 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c21525-52dc-4046-92b7-191abb2e8050 admin admin] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:26:03.797544 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c21525-52dc-4046-92b7-191abb2e8050 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:26:03.798113 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c21525-52dc-4046-92b7-191abb2e8050 admin admin] PATH_INFO: `/v3/users` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:26:03.798758 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c21525-52dc-4046-92b7-191abb2e8050 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:26:03.800706 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c21525-52dc-4046-92b7-191abb2e8050 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:26:03.825609 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1111/2248] 10.222.0.15 () {60 vars in 1144 bytes} [Thu Sep 15 12:26:03 2022] POST /identity/v3/users => generated 289 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 12:26:03.826922 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c433c8b0-fbb5-439e-adc0-79e957414438 admin admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:26:03.837254 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c433c8b0-fbb5-439e-adc0-79e957414438 admin admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:26:03.841829 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb938a0-88d4-44d1-8b73-6c12e4fea17a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:26:03.842525 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb938a0-88d4-44d1-8b73-6c12e4fea17a admin admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:26:03.842860 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb938a0-88d4-44d1-8b73-6c12e4fea17a admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:26:03.843189 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb938a0-88d4-44d1-8b73-6c12e4fea17a admin admin] PATH_INFO: `/v3/roles` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:26:03.844390 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb938a0-88d4-44d1-8b73-6c12e4fea17a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:26:03.845848 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb938a0-88d4-44d1-8b73-6c12e4fea17a admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:26:03.857457 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1138/2249] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 12:26:03 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 12:26:03.861517 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c21525-52dc-4046-92b7-191abb2e8050 admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:26:03.875543 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c21525-52dc-4046-92b7-191abb2e8050 admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:26:03.880068 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e744859-8de8-4c73-bebd-3a27db90faf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:26:03.880662 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e744859-8de8-4c73-bebd-3a27db90faf8 admin admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:26:03.880988 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e744859-8de8-4c73-bebd-3a27db90faf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:26:03.881258 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e744859-8de8-4c73-bebd-3a27db90faf8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:26:03.881881 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e744859-8de8-4c73-bebd-3a27db90faf8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:26:03.883465 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e744859-8de8-4c73-bebd-3a27db90faf8 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:26:03.895023 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1112/2250] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 12:26:03 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 12:26:03.899309 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb938a0-88d4-44d1-8b73-6c12e4fea17a admin admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:26:03.909744 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb938a0-88d4-44d1-8b73-6c12e4fea17a admin admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:26:03.914533 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435e9de4-fb87-4d27-9a42-538ef3d9ab22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:26:03.915184 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435e9de4-fb87-4d27-9a42-538ef3d9ab22 admin admin] REQUEST_METHOD: `PUT` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:26:03.915461 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435e9de4-fb87-4d27-9a42-538ef3d9ab22 admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:26:03.915739 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435e9de4-fb87-4d27-9a42-538ef3d9ab22 admin admin] PATH_INFO: `/v3/projects/1e689c16957541128374b5a9e15e24da/users/de5a84327d8645af9c7b15d48161b120/roles/0679d6628a25438d90dc4bdc0401eda2` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:26:03.936034 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435e9de4-fb87-4d27-9a42-538ef3d9ab22 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1e689c16957541128374b5a9e15e24da, user_id=de5a84327d8645af9c7b15d48161b120, role_id=0679d6628a25438d90dc4bdc0401eda2)` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:26:03.938751 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435e9de4-fb87-4d27-9a42-538ef3d9ab22 admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:26:03.951214 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1139/2251] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 15 12:26:03 2022] PUT /identity/v3/projects/1e689c16957541128374b5a9e15e24da/users/de5a84327d8645af9c7b15d48161b120/roles/0679d6628a25438d90dc4bdc0401eda2 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:26:03.961380 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21119db9-c0a9-4d36-ba84-57c17d3bb6f9 None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:26:03.961664 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21119db9-c0a9-4d36-ba84-57c17d3bb6f9 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:26:03.961933 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21119db9-c0a9-4d36-ba84-57c17d3bb6f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:26:03.991066 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-21119db9-c0a9-4d36-ba84-57c17d3bb6f9 None None] MFA Rules not processed for user `de5a84327d8645af9c7b15d48161b120`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:26:03.992311 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-21119db9-c0a9-4d36-ba84-57c17d3bb6f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:26:04.054607 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1113/2252] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 12:26:03 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:26:04.065854 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435e9de4-fb87-4d27-9a42-538ef3d9ab22 admin admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:26:04.067556 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-435e9de4-fb87-4d27-9a42-538ef3d9ab22 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:26:04.074026 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:26:04.125475 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435e9de4-fb87-4d27-9a42-538ef3d9ab22 admin admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:26:04.129044 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e73af42-e82b-4ebf-a85d-4eb16a9799b7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8191d3ccb874529b32f8d0026db629d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '540a32ef88764e8c8d9201ae22f6ddaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:26:04.129643 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e73af42-e82b-4ebf-a85d-4eb16a9799b7 service cinder] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:26:04.129923 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e73af42-e82b-4ebf-a85d-4eb16a9799b7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:26:04.130193 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e73af42-e82b-4ebf-a85d-4eb16a9799b7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:26:04.138251 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e73af42-e82b-4ebf-a85d-4eb16a9799b7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:26:04.139664 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e73af42-e82b-4ebf-a85d-4eb16a9799b7 service cinder] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:26:04.148212 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1140/2253] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 12:26:04 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 12:26:14.051744 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=43,keystone:INSERT=3 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:26:14.144249 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=24 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:26:22.219479 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf3f985-4702-4dfa-861c-f746a7a604f1 None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:26:22.219998 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf3f985-4702-4dfa-861c-f746a7a604f1 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:26:22.220090 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf3f985-4702-4dfa-861c-f746a7a604f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:26:22.238413 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-faf3f985-4702-4dfa-861c-f746a7a604f1 None None] MFA Rules not processed for user `4ead4be9666942f2b81cbb0f9ab69325`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:26:22.239632 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-faf3f985-4702-4dfa-861c-f746a7a604f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:26:22.292557 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1114/2254] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 12:26:22 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:26:22.852280 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e73af42-e82b-4ebf-a85d-4eb16a9799b7 service cinder] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:26:22.916020 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e73af42-e82b-4ebf-a85d-4eb16a9799b7 service cinder] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:26:22.920564 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680c7094-f0d2-411f-a8f6-97f733ad815d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:26:22.921234 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680c7094-f0d2-411f-a8f6-97f733ad815d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:26:22.921513 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680c7094-f0d2-411f-a8f6-97f733ad815d admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:26:22.921781 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680c7094-f0d2-411f-a8f6-97f733ad815d admin admin] PATH_INFO: `/v3/users/de5a84327d8645af9c7b15d48161b120` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:26:22.923067 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680c7094-f0d2-411f-a8f6-97f733ad815d admin admin] RBAC: Authorizing `identity:delete_user(user_id=de5a84327d8645af9c7b15d48161b120)` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:26:22.924996 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680c7094-f0d2-411f-a8f6-97f733ad815d admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:26:22.991095 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-680c7094-f0d2-411f-a8f6-97f733ad815d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'de5a84327d8645af9c7b15d48161b120'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:26:23.001502 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-680c7094-f0d2-411f-a8f6-97f733ad815d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'de5a84327d8645af9c7b15d48161b120'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:26:23.017067 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-680c7094-f0d2-411f-a8f6-97f733ad815d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'de5a84327d8645af9c7b15d48161b120'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:26:23.017858 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-680c7094-f0d2-411f-a8f6-97f733ad815d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'de5a84327d8645af9c7b15d48161b120'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:26:23.026841 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-680c7094-f0d2-411f-a8f6-97f733ad815d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'de5a84327d8645af9c7b15d48161b120'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:26:23.033449 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1141/2255] 10.222.0.15 () {58 vars in 1224 bytes} [Thu Sep 15 12:26:22 2022] DELETE /identity/v3/users/de5a84327d8645af9c7b15d48161b120 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:26:23.037932 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7481b20b-8fd6-41f1-8316-8d082347c6da None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:26:23.038292 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7481b20b-8fd6-41f1-8316-8d082347c6da None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:26:23.038635 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7481b20b-8fd6-41f1-8316-8d082347c6da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:26:23.065204 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-7481b20b-8fd6-41f1-8316-8d082347c6da None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:26:23.067396 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-7481b20b-8fd6-41f1-8316-8d082347c6da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:26:23.168807 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1115/2256] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 12:26:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 12:26:23.177327 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680c7094-f0d2-411f-a8f6-97f733ad815d admin admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:26:23.178725 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-680c7094-f0d2-411f-a8f6-97f733ad815d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:26:23.241486 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680c7094-f0d2-411f-a8f6-97f733ad815d admin admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:26:23.245092 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7606e5ec-2f5a-4700-bd7d-720c56d990ef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfbb4d6809064f9cb9e06626ed7d6814', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '540a32ef88764e8c8d9201ae22f6ddaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:26:23.245696 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7606e5ec-2f5a-4700-bd7d-720c56d990ef service neutron] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:26:23.245980 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7606e5ec-2f5a-4700-bd7d-720c56d990ef service neutron] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:26:23.246252 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7606e5ec-2f5a-4700-bd7d-720c56d990ef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:26:23.255751 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7606e5ec-2f5a-4700-bd7d-720c56d990ef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:26:23.257271 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7606e5ec-2f5a-4700-bd7d-720c56d990ef service neutron] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:26:23.267872 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1142/2257] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 12:26:23 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 12:26:23.548993 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7481b20b-8fd6-41f1-8316-8d082347c6da None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:26:23.550505 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-7481b20b-8fd6-41f1-8316-8d082347c6da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:26:23.569277 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7481b20b-8fd6-41f1-8316-8d082347c6da None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:26:23.575071 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a81bae-b5af-4e4c-b8dd-7b327aa80d1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:26:23.575853 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a81bae-b5af-4e4c-b8dd-7b327aa80d1e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:26:23.576233 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a81bae-b5af-4e4c-b8dd-7b327aa80d1e admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:26:23.576986 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a81bae-b5af-4e4c-b8dd-7b327aa80d1e admin admin] PATH_INFO: `/v3/projects/1e689c16957541128374b5a9e15e24da` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:26:23.578653 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a81bae-b5af-4e4c-b8dd-7b327aa80d1e admin admin] RBAC: Authorizing `identity:delete_project(project_id=1e689c16957541128374b5a9e15e24da)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:26:23.581284 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a81bae-b5af-4e4c-b8dd-7b327aa80d1e admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:26:23.630011 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-45a81bae-b5af-4e4c-b8dd-7b327aa80d1e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1e689c16957541128374b5a9e15e24da', 'request_id': 'req-45a81bae-b5af-4e4c-b8dd-7b327aa80d1e'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:26:23.637715 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-45a81bae-b5af-4e4c-b8dd-7b327aa80d1e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1e689c16957541128374b5a9e15e24da', 'request_id': 'req-45a81bae-b5af-4e4c-b8dd-7b327aa80d1e'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:26:23.643817 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-45a81bae-b5af-4e4c-b8dd-7b327aa80d1e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1e689c16957541128374b5a9e15e24da', 'request_id': 'req-45a81bae-b5af-4e4c-b8dd-7b327aa80d1e'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:26:23.655682 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-45a81bae-b5af-4e4c-b8dd-7b327aa80d1e admin admin] The token cache is being invalidate because project 1e689c16957541128374b5a9e15e24da was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81448) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 12:26:23.655971 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-45a81bae-b5af-4e4c-b8dd-7b327aa80d1e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:26:23.656686 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-45a81bae-b5af-4e4c-b8dd-7b327aa80d1e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:26:23.658356 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1116/2258] 10.222.0.15 () {58 vars in 1233 bytes} [Thu Sep 15 12:26:23 2022] DELETE /identity/v3/projects/1e689c16957541128374b5a9e15e24da => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 12:26:23.663585 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1079bf-7f6c-40af-a146-943c6177d24c None None] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:26:23.663951 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1079bf-7f6c-40af-a146-943c6177d24c None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:26:23.664230 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1079bf-7f6c-40af-a146-943c6177d24c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:26:23.684857 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.auth.core [None req-ae1079bf-7f6c-40af-a146-943c6177d24c None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81449) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:26:23.686068 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-ae1079bf-7f6c-40af-a146-943c6177d24c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:26:23.704080 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:26:23.754149 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1143/2259] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 12:26:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:26:23.763274 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5a2cf1-a44e-4912-8e1a-fc8815e30879 None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:26:23.763570 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5a2cf1-a44e-4912-8e1a-fc8815e30879 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:26:23.763845 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5a2cf1-a44e-4912-8e1a-fc8815e30879 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:26:23.787832 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-2a5a2cf1-a44e-4912-8e1a-fc8815e30879 None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:26:23.789087 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-2a5a2cf1-a44e-4912-8e1a-fc8815e30879 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:26:23.800350 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1117/2260] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 12:26:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:26:23.805039 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1079bf-7f6c-40af-a146-943c6177d24c None None] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:26:23.816000 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1079bf-7f6c-40af-a146-943c6177d24c None None] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:26:23.821403 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c916aa-8d7e-408b-a544-c8147fc3acf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:26:23.822035 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c916aa-8d7e-408b-a544-c8147fc3acf4 admin admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:26:23.822996 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c916aa-8d7e-408b-a544-c8147fc3acf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:26:23.823279 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c916aa-8d7e-408b-a544-c8147fc3acf4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:26:23.823928 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c916aa-8d7e-408b-a544-c8147fc3acf4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81449) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:26:23.824233 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c916aa-8d7e-408b-a544-c8147fc3acf4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:26:23.825885 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c916aa-8d7e-408b-a544-c8147fc3acf4 admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:26:23.842854 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1144/2261] 10.222.0.15 () {58 vars in 1153 bytes} [Thu Sep 15 12:26:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:26:23.844591 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5a2cf1-a44e-4912-8e1a-fc8815e30879 None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:26:23.855544 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5a2cf1-a44e-4912-8e1a-fc8815e30879 None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:26:23.860384 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77b63b5-ebec-43f1-937d-13b7f01418a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:26:23.861108 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77b63b5-ebec-43f1-937d-13b7f01418a0 admin admin] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:26:23.861457 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77b63b5-ebec-43f1-937d-13b7f01418a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:26:23.861836 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77b63b5-ebec-43f1-937d-13b7f01418a0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:26:23.862539 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c77b63b5-ebec-43f1-937d-13b7f01418a0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:26:23.864512 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c77b63b5-ebec-43f1-937d-13b7f01418a0 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:26:23.895651 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:26:23.905432 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1118/2262] 10.222.0.15 () {60 vars in 1153 bytes} [Thu Sep 15 12:26:23 2022] POST /identity/v3/projects => generated 375 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 12:26:23.909467 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c916aa-8d7e-408b-a544-c8147fc3acf4 admin admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:26:23.920657 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c916aa-8d7e-408b-a544-c8147fc3acf4 admin admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:26:23.976860 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91f9555-0256-4cee-8629-6e2d181bd9b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:26:23.977486 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91f9555-0256-4cee-8629-6e2d181bd9b5 admin admin] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:26:23.977768 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91f9555-0256-4cee-8629-6e2d181bd9b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:26:23.978069 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91f9555-0256-4cee-8629-6e2d181bd9b5 admin admin] PATH_INFO: `/v3/users` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:26:23.978733 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91f9555-0256-4cee-8629-6e2d181bd9b5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:26:23.980879 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91f9555-0256-4cee-8629-6e2d181bd9b5 admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:26:24.002924 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1145/2263] 10.222.0.15 () {60 vars in 1144 bytes} [Thu Sep 15 12:26:23 2022] POST /identity/v3/users => generated 288 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 15 12:26:24.007501 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77b63b5-ebec-43f1-937d-13b7f01418a0 admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:26:24.018864 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77b63b5-ebec-43f1-937d-13b7f01418a0 admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:26:24.028500 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976a8ab4-879c-42e3-8027-905a6a2eb240 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:26:24.029150 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976a8ab4-879c-42e3-8027-905a6a2eb240 admin admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:26:24.029428 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976a8ab4-879c-42e3-8027-905a6a2eb240 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:26:24.029700 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976a8ab4-879c-42e3-8027-905a6a2eb240 admin admin] PATH_INFO: `/v3/roles` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:26:24.030341 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-976a8ab4-879c-42e3-8027-905a6a2eb240 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:26:24.031941 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-976a8ab4-879c-42e3-8027-905a6a2eb240 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:26:24.042378 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1119/2264] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 12:26:24 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 12:26:24.046216 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91f9555-0256-4cee-8629-6e2d181bd9b5 admin admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:26:24.056281 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91f9555-0256-4cee-8629-6e2d181bd9b5 admin admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:26:24.061154 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b52de07-c317-436f-9ac0-578926ac05a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:26:24.061904 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b52de07-c317-436f-9ac0-578926ac05a3 admin admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:26:24.062187 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b52de07-c317-436f-9ac0-578926ac05a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:26:24.062588 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b52de07-c317-436f-9ac0-578926ac05a3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:26:24.063238 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b52de07-c317-436f-9ac0-578926ac05a3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:26:24.064856 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b52de07-c317-436f-9ac0-578926ac05a3 admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:26:24.076237 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1146/2265] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 12:26:24 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 12:26:24.080408 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976a8ab4-879c-42e3-8027-905a6a2eb240 admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:26:24.091080 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976a8ab4-879c-42e3-8027-905a6a2eb240 admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:26:24.095879 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ed05c1-c601-4d16-b68b-4153f3bd3e8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:26:24.096539 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ed05c1-c601-4d16-b68b-4153f3bd3e8b admin admin] REQUEST_METHOD: `PUT` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:26:24.096845 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ed05c1-c601-4d16-b68b-4153f3bd3e8b admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:26:24.097117 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ed05c1-c601-4d16-b68b-4153f3bd3e8b admin admin] PATH_INFO: `/v3/projects/05528656883142d98e5cab82b7f11e7c/users/01c9af0c4c8946b685b22474efb139b1/roles/0679d6628a25438d90dc4bdc0401eda2` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:26:24.111699 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ed05c1-c601-4d16-b68b-4153f3bd3e8b admin admin] RBAC: Authorizing `identity:create_grant(project_id=05528656883142d98e5cab82b7f11e7c, user_id=01c9af0c4c8946b685b22474efb139b1, role_id=0679d6628a25438d90dc4bdc0401eda2)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:26:24.114392 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ed05c1-c601-4d16-b68b-4153f3bd3e8b admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:26:24.127307 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1120/2266] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 15 12:26:24 2022] PUT /identity/v3/projects/05528656883142d98e5cab82b7f11e7c/users/01c9af0c4c8946b685b22474efb139b1/roles/0679d6628a25438d90dc4bdc0401eda2 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 12:26:24.138718 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e03ffce-edb6-48ac-99e5-5053770157f3 None None] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:26:24.139132 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e03ffce-edb6-48ac-99e5-5053770157f3 None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:26:24.139613 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e03ffce-edb6-48ac-99e5-5053770157f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:26:24.169166 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.auth.core [None req-7e03ffce-edb6-48ac-99e5-5053770157f3 None None] MFA Rules not processed for user `01c9af0c4c8946b685b22474efb139b1`. Rule list: `[]` (Enabled: `True`). {{(pid=81449) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:26:24.170357 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-7e03ffce-edb6-48ac-99e5-5053770157f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:26:24.256998 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1147/2267] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 12:26:24 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:26:24.272267 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ed05c1-c601-4d16-b68b-4153f3bd3e8b admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:26:24.274720 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-70ed05c1-c601-4d16-b68b-4153f3bd3e8b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:26:24.345929 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ed05c1-c601-4d16-b68b-4153f3bd3e8b admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:26:24.353032 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec938974-2a4d-48bb-b947-2d03cf7b0140 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8191d3ccb874529b32f8d0026db629d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '540a32ef88764e8c8d9201ae22f6ddaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:26:24.357149 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec938974-2a4d-48bb-b947-2d03cf7b0140 service cinder] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:26:24.357149 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec938974-2a4d-48bb-b947-2d03cf7b0140 service cinder] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:26:24.357149 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec938974-2a4d-48bb-b947-2d03cf7b0140 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:26:24.382309 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec938974-2a4d-48bb-b947-2d03cf7b0140 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:26:24.384283 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec938974-2a4d-48bb-b947-2d03cf7b0140 service cinder] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:26:24.395045 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1121/2268] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 12:26:24 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 12:26:25.942505 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35915ccb-6a5d-40fa-bb5d-b634598e59b4 None None] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:26:25.942985 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35915ccb-6a5d-40fa-bb5d-b634598e59b4 None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:26:25.943621 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35915ccb-6a5d-40fa-bb5d-b634598e59b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:26:25.968849 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.auth.core [None req-35915ccb-6a5d-40fa-bb5d-b634598e59b4 None None] MFA Rules not processed for user `527f4dd6031546c785bc6360dacb60af`. Rule list: `[]` (Enabled: `True`). {{(pid=81449) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:26:25.970196 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-35915ccb-6a5d-40fa-bb5d-b634598e59b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:26:25.992029 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:26:26.049280 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1148/2269] 10.222.0.15 () {58 vars in 1009 bytes} [Thu Sep 15 12:26:25 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 12:26:26.103988 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec938974-2a4d-48bb-b947-2d03cf7b0140 service cinder] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:26:26.105790 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-ec938974-2a4d-48bb-b947-2d03cf7b0140 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:26:26.238343 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec938974-2a4d-48bb-b947-2d03cf7b0140 service cinder] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:26:26.245582 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5727ac-0f3a-4fca-b54b-fd7362e94533 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa0d3dc5b7540d59fdd59bbf64a04b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '540a32ef88764e8c8d9201ae22f6ddaf', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:26:26.246249 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5727ac-0f3a-4fca-b54b-fd7362e94533 service swift] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:26:26.246514 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5727ac-0f3a-4fca-b54b-fd7362e94533 service swift] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:26:26.247079 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5727ac-0f3a-4fca-b54b-fd7362e94533 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:26:26.259263 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5727ac-0f3a-4fca-b54b-fd7362e94533 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:26:26.262068 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5727ac-0f3a-4fca-b54b-fd7362e94533 service swift] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:26:26.296286 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1122/2270] 10.222.0.15 () {60 vars in 1384 bytes} [Thu Sep 15 12:26:26 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 193 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 15 12:26:35.063182 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35915ccb-6a5d-40fa-bb5d-b634598e59b4 None None] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:26:35.136474 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35915ccb-6a5d-40fa-bb5d-b634598e59b4 None None] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:26:35.141792 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3f1262-3d9a-468e-9e13-1e9e5832423b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:26:35.142454 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3f1262-3d9a-468e-9e13-1e9e5832423b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:26:35.142727 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3f1262-3d9a-468e-9e13-1e9e5832423b admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:26:35.142995 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3f1262-3d9a-468e-9e13-1e9e5832423b admin admin] PATH_INFO: `/v3/users/01c9af0c4c8946b685b22474efb139b1` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:26:35.144301 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3f1262-3d9a-468e-9e13-1e9e5832423b admin admin] RBAC: Authorizing `identity:delete_user(user_id=01c9af0c4c8946b685b22474efb139b1)` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:26:35.146269 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3f1262-3d9a-468e-9e13-1e9e5832423b admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:26:35.201861 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-cc3f1262-3d9a-468e-9e13-1e9e5832423b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '01c9af0c4c8946b685b22474efb139b1'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:26:35.215930 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-cc3f1262-3d9a-468e-9e13-1e9e5832423b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '01c9af0c4c8946b685b22474efb139b1'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:26:35.230726 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-cc3f1262-3d9a-468e-9e13-1e9e5832423b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '01c9af0c4c8946b685b22474efb139b1'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:26:35.231503 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-cc3f1262-3d9a-468e-9e13-1e9e5832423b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '01c9af0c4c8946b685b22474efb139b1'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:26:35.240120 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-cc3f1262-3d9a-468e-9e13-1e9e5832423b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '01c9af0c4c8946b685b22474efb139b1'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:26:35.248809 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1149/2271] 10.222.0.15 () {58 vars in 1224 bytes} [Thu Sep 15 12:26:35 2022] DELETE /identity/v3/users/01c9af0c4c8946b685b22474efb139b1 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 12:26:35.253235 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971df2c7-10e5-4599-b4ea-67a9f41f21af None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:26:35.253756 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971df2c7-10e5-4599-b4ea-67a9f41f21af None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:26:35.253756 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971df2c7-10e5-4599-b4ea-67a9f41f21af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:26:35.277256 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-971df2c7-10e5-4599-b4ea-67a9f41f21af None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:26:35.278635 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-971df2c7-10e5-4599-b4ea-67a9f41f21af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:26:35.367515 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:26:35.385492 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1123/2272] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 12:26:35 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:26:35.395806 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3f1262-3d9a-468e-9e13-1e9e5832423b admin admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:26:35.397605 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-cc3f1262-3d9a-468e-9e13-1e9e5832423b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:26:35.486043 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:26:35.514160 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3f1262-3d9a-468e-9e13-1e9e5832423b admin admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:26:35.517992 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967d1bb5-8009-459d-9455-31e9c260e687 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfbb4d6809064f9cb9e06626ed7d6814', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '540a32ef88764e8c8d9201ae22f6ddaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:26:35.518717 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967d1bb5-8009-459d-9455-31e9c260e687 service neutron] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:26:35.519125 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967d1bb5-8009-459d-9455-31e9c260e687 service neutron] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:26:35.519497 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967d1bb5-8009-459d-9455-31e9c260e687 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:26:35.531119 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-967d1bb5-8009-459d-9455-31e9c260e687 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:26:35.533298 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-967d1bb5-8009-459d-9455-31e9c260e687 service neutron] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:26:35.549035 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1150/2273] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 12:26:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 12:26:36.064145 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971df2c7-10e5-4599-b4ea-67a9f41f21af None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:26:36.068007 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-971df2c7-10e5-4599-b4ea-67a9f41f21af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:26:36.081461 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971df2c7-10e5-4599-b4ea-67a9f41f21af None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:26:36.086498 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0957d39-20df-4681-8d60-1d26898aa3fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:26:36.087218 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0957d39-20df-4681-8d60-1d26898aa3fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:26:36.087451 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0957d39-20df-4681-8d60-1d26898aa3fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:26:36.087719 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0957d39-20df-4681-8d60-1d26898aa3fc admin admin] PATH_INFO: `/v3/projects/05528656883142d98e5cab82b7f11e7c` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:26:36.089054 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0957d39-20df-4681-8d60-1d26898aa3fc admin admin] RBAC: Authorizing `identity:delete_project(project_id=05528656883142d98e5cab82b7f11e7c)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:26:36.100874 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0957d39-20df-4681-8d60-1d26898aa3fc admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:26:36.176927 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-f0957d39-20df-4681-8d60-1d26898aa3fc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '05528656883142d98e5cab82b7f11e7c', 'request_id': 'req-f0957d39-20df-4681-8d60-1d26898aa3fc'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:26:36.188257 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-f0957d39-20df-4681-8d60-1d26898aa3fc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '05528656883142d98e5cab82b7f11e7c', 'request_id': 'req-f0957d39-20df-4681-8d60-1d26898aa3fc'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:26:36.207284 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-f0957d39-20df-4681-8d60-1d26898aa3fc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '05528656883142d98e5cab82b7f11e7c', 'request_id': 'req-f0957d39-20df-4681-8d60-1d26898aa3fc'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:26:36.220145 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-f0957d39-20df-4681-8d60-1d26898aa3fc admin admin] The token cache is being invalidate because project 05528656883142d98e5cab82b7f11e7c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81448) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 12:26:36.220440 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-f0957d39-20df-4681-8d60-1d26898aa3fc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:26:36.221136 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-f0957d39-20df-4681-8d60-1d26898aa3fc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:26:36.222789 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1124/2274] 10.222.0.15 () {58 vars in 1233 bytes} [Thu Sep 15 12:26:36 2022] DELETE /identity/v3/projects/05528656883142d98e5cab82b7f11e7c => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 12:26:41.235786 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47c7f0c-3184-44a4-b274-879b197407a9 None None] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:26:41.236377 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47c7f0c-3184-44a4-b274-879b197407a9 None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:26:41.236377 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47c7f0c-3184-44a4-b274-879b197407a9 None None] PATH_INFO: `/` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:26:41.238199 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1151/2275] 10.222.0.15 () {54 vars in 921 bytes} [Thu Sep 15 12:26:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:26:41.244432 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af002b8-82cc-4f3d-a2c2-f6b8b45b41d2 None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:26:41.244972 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af002b8-82cc-4f3d-a2c2-f6b8b45b41d2 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:26:41.245078 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af002b8-82cc-4f3d-a2c2-f6b8b45b41d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:26:41.277480 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-9af002b8-82cc-4f3d-a2c2-f6b8b45b41d2 None None] MFA Rules not processed for user `eface89e261840a2abb4206be26455c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:26:41.278677 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-9af002b8-82cc-4f3d-a2c2-f6b8b45b41d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:26:41.354041 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1125/2276] 10.222.0.15 () {58 vars in 1018 bytes} [Thu Sep 15 12:26:41 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:26:41.365631 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47c7f0c-3184-44a4-b274-879b197407a9 None None] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:26:41.367622 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-e47c7f0c-3184-44a4-b274-879b197407a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:26:41.430568 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47c7f0c-3184-44a4-b274-879b197407a9 None None] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:26:41.434688 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3cb15e-1550-4ff2-b3e6-4c68b190746a service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3aed8f78903f44cda9e5254aa46cf2b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '540a32ef88764e8c8d9201ae22f6ddaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:26:41.435295 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3cb15e-1550-4ff2-b3e6-4c68b190746a service glance] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:26:41.435577 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3cb15e-1550-4ff2-b3e6-4c68b190746a service glance] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:26:41.435848 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3cb15e-1550-4ff2-b3e6-4c68b190746a service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:26:41.446847 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f3cb15e-1550-4ff2-b3e6-4c68b190746a service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:26:41.452963 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f3cb15e-1550-4ff2-b3e6-4c68b190746a service glance] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:26:41.459430 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1152/2277] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 12:26:41 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 12:26:41.530091 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7285439f-8d51-4469-b278-aec8f2e20ba9 None None] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:26:41.530376 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7285439f-8d51-4469-b278-aec8f2e20ba9 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:26:41.530667 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7285439f-8d51-4469-b278-aec8f2e20ba9 None None] PATH_INFO: `/` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:26:41.532035 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1126/2278] 10.222.0.15 () {54 vars in 921 bytes} [Thu Sep 15 12:26:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:26:41.536600 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2aae840-1a99-459a-adaa-19dda2ceddba None None] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:26:41.536915 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2aae840-1a99-459a-adaa-19dda2ceddba None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:26:41.537193 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2aae840-1a99-459a-adaa-19dda2ceddba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:26:41.557007 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.auth.core [None req-f2aae840-1a99-459a-adaa-19dda2ceddba None None] MFA Rules not processed for user `bfbb4d6809064f9cb9e06626ed7d6814`. Rule list: `[]` (Enabled: `True`). {{(pid=81449) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:26:41.558196 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-f2aae840-1a99-459a-adaa-19dda2ceddba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:26:41.608100 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1153/2279] 10.222.0.15 () {58 vars in 1018 bytes} [Thu Sep 15 12:26:41 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 72 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:26:41.620034 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7285439f-8d51-4469-b278-aec8f2e20ba9 None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:26:41.621737 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-7285439f-8d51-4469-b278-aec8f2e20ba9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:26:41.634457 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7285439f-8d51-4469-b278-aec8f2e20ba9 None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:26:41.637908 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02b1222-3195-4ffb-8ba7-a2031dba0521 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfbb4d6809064f9cb9e06626ed7d6814', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '540a32ef88764e8c8d9201ae22f6ddaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:26:41.638498 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02b1222-3195-4ffb-8ba7-a2031dba0521 service neutron] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:26:41.638775 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02b1222-3195-4ffb-8ba7-a2031dba0521 service neutron] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:26:41.639045 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02b1222-3195-4ffb-8ba7-a2031dba0521 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:26:41.645852 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02b1222-3195-4ffb-8ba7-a2031dba0521 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:26:41.647255 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02b1222-3195-4ffb-8ba7-a2031dba0521 service neutron] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:26:41.655454 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1127/2280] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 12:26:41 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 12:26:41.771789 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c95497d-6fb0-4687-a05a-95a9998d0328 None None] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:26:41.772077 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c95497d-6fb0-4687-a05a-95a9998d0328 None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:26:41.772378 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c95497d-6fb0-4687-a05a-95a9998d0328 None None] PATH_INFO: `/` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:26:41.773751 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1154/2281] 10.222.0.15 () {54 vars in 921 bytes} [Thu Sep 15 12:26:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Sep 15 12:26:41.779564 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91847fc1-e9bd-4057-b987-7e508cc9db5f None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:26:41.779849 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91847fc1-e9bd-4057-b987-7e508cc9db5f None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:26:41.780123 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91847fc1-e9bd-4057-b987-7e508cc9db5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:26:41.799737 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-91847fc1-e9bd-4057-b987-7e508cc9db5f None None] MFA Rules not processed for user `eface89e261840a2abb4206be26455c7`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:26:41.800957 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-91847fc1-e9bd-4057-b987-7e508cc9db5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:26:41.810310 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1128/2282] 10.222.0.15 () {58 vars in 1018 bytes} [Thu Sep 15 12:26:41 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:26:41.821880 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c95497d-6fb0-4687-a05a-95a9998d0328 None None] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:26:41.832587 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c95497d-6fb0-4687-a05a-95a9998d0328 None None] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:26:41.836030 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885e96ec-1116-40a6-9575-0f5234f111a8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfbb4d6809064f9cb9e06626ed7d6814', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '540a32ef88764e8c8d9201ae22f6ddaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:26:41.836618 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885e96ec-1116-40a6-9575-0f5234f111a8 service neutron] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:26:41.837447 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885e96ec-1116-40a6-9575-0f5234f111a8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:26:41.837726 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-885e96ec-1116-40a6-9575-0f5234f111a8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:26:41.847727 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-885e96ec-1116-40a6-9575-0f5234f111a8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:26:41.849187 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-885e96ec-1116-40a6-9575-0f5234f111a8 service neutron] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:26:41.859532 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1155/2283] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 12:26:41 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 12:26:44.238967 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e91622-e047-41f1-b3aa-a37ad6846547 None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:26:44.239455 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e91622-e047-41f1-b3aa-a37ad6846547 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:26:44.239557 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e91622-e047-41f1-b3aa-a37ad6846547 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:26:44.259962 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-c7e91622-e047-41f1-b3aa-a37ad6846547 None None] MFA Rules not processed for user `527f4dd6031546c785bc6360dacb60af`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:26:44.261431 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-c7e91622-e047-41f1-b3aa-a37ad6846547 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:26:44.308178 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:26:44.319382 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1129/2284] 10.222.0.15 () {58 vars in 1009 bytes} [Thu Sep 15 12:26:44 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:26:44.332017 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885e96ec-1116-40a6-9575-0f5234f111a8 service neutron] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:26:44.333443 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-885e96ec-1116-40a6-9575-0f5234f111a8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:26:44.401390 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-885e96ec-1116-40a6-9575-0f5234f111a8 service neutron] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:26:44.406170 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113e8cdb-57d8-461f-b46a-c2f05bb6e181 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa0d3dc5b7540d59fdd59bbf64a04b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '540a32ef88764e8c8d9201ae22f6ddaf', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:26:44.406770 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113e8cdb-57d8-461f-b46a-c2f05bb6e181 service swift] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:26:44.407046 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113e8cdb-57d8-461f-b46a-c2f05bb6e181 service swift] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:26:44.407314 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113e8cdb-57d8-461f-b46a-c2f05bb6e181 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:26:44.415023 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-113e8cdb-57d8-461f-b46a-c2f05bb6e181 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:26:44.416600 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-113e8cdb-57d8-461f-b46a-c2f05bb6e181 service swift] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:26:44.424356 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1156/2285] 10.222.0.15 () {60 vars in 1384 bytes} [Thu Sep 15 12:26:44 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 15 12:26:54.320218 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=1 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:26:54.421273 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=94 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:27:05.450395 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c95416-e44c-4dc1-9b2f-cd75f1220781 None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:27:05.450865 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c95416-e44c-4dc1-9b2f-cd75f1220781 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:27:05.450968 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c95416-e44c-4dc1-9b2f-cd75f1220781 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:27:05.471160 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-48c95416-e44c-4dc1-9b2f-cd75f1220781 None None] MFA Rules not processed for user `20ca976222454c6c898d0e4b170b41fc`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:27:05.472405 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-48c95416-e44c-4dc1-9b2f-cd75f1220781 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:27:05.528636 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1130/2286] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 12:27:05 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 12:27:09.783700 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8760d763-1f83-465c-bfd9-068b39a43412 None None] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:27:09.784173 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8760d763-1f83-465c-bfd9-068b39a43412 None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:27:09.784265 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8760d763-1f83-465c-bfd9-068b39a43412 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:27:09.805382 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.auth.core [None req-8760d763-1f83-465c-bfd9-068b39a43412 None None] MFA Rules not processed for user `527f4dd6031546c785bc6360dacb60af`. Rule list: `[]` (Enabled: `True`). {{(pid=81449) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:27:09.806843 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-8760d763-1f83-465c-bfd9-068b39a43412 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:27:09.819217 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1157/2287] 10.222.0.15 () {58 vars in 967 bytes} [Thu Sep 15 12:27:09 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:27:09.827300 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c95416-e44c-4dc1-9b2f-cd75f1220781 None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:27:09.839998 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c95416-e44c-4dc1-9b2f-cd75f1220781 None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:27:09.847113 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1588cf70-c517-466c-a7e5-f4008e648fdb service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa0d3dc5b7540d59fdd59bbf64a04b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '540a32ef88764e8c8d9201ae22f6ddaf', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:27:09.848005 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1588cf70-c517-466c-a7e5-f4008e648fdb service swift] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:27:09.848287 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1588cf70-c517-466c-a7e5-f4008e648fdb service swift] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:27:09.848561 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1588cf70-c517-466c-a7e5-f4008e648fdb service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:27:09.861404 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1588cf70-c517-466c-a7e5-f4008e648fdb service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:27:09.862958 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1588cf70-c517-466c-a7e5-f4008e648fdb service swift] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:27:09.872161 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1131/2288] 10.222.0.15 () {60 vars in 1384 bytes} [Thu Sep 15 12:27:09 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 15 12:27:16.604933 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8760d763-1f83-465c-bfd9-068b39a43412 None None] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:27:16.606344 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-8760d763-1f83-465c-bfd9-068b39a43412 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:27:16.680238 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8760d763-1f83-465c-bfd9-068b39a43412 None None] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:27:16.686225 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debe83eb-14aa-4ca6-ab60-d53cea0132da service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f64a058bd5440cb5268f228aa6e392', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '540a32ef88764e8c8d9201ae22f6ddaf', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:27:16.686837 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debe83eb-14aa-4ca6-ab60-d53cea0132da service placement] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:27:16.687115 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debe83eb-14aa-4ca6-ab60-d53cea0132da service placement] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:27:16.687380 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debe83eb-14aa-4ca6-ab60-d53cea0132da service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:27:16.689159 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-debe83eb-14aa-4ca6-ab60-d53cea0132da service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:27:16.700833 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-debe83eb-14aa-4ca6-ab60-d53cea0132da service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:27:16.702425 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-debe83eb-14aa-4ca6-ab60-d53cea0132da service placement] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:27:16.712919 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1158/2289] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 12:27:16 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 12:27:19.869239 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=31 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:27:26.712686 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=35 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:27:34.410571 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7677f8c5-78bf-47db-b126-bfa5a710fd50 None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:27:34.411174 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7677f8c5-78bf-47db-b126-bfa5a710fd50 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:27:34.411174 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7677f8c5-78bf-47db-b126-bfa5a710fd50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:27:34.431026 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-7677f8c5-78bf-47db-b126-bfa5a710fd50 None None] MFA Rules not processed for user `4ead4be9666942f2b81cbb0f9ab69325`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:27:34.432214 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-7677f8c5-78bf-47db-b126-bfa5a710fd50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:27:34.485779 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1132/2290] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 12:27:34 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:27:44.479335 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=25 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:28:00.138803 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3452afd0-913d-4905-b5e2-6d2a42f70a28 None None] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:28:00.139322 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3452afd0-913d-4905-b5e2-6d2a42f70a28 None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:28:00.139451 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3452afd0-913d-4905-b5e2-6d2a42f70a28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:28:00.159792 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.auth.core [None req-3452afd0-913d-4905-b5e2-6d2a42f70a28 None None] MFA Rules not processed for user `1c42d5435a514bc0880b69fbbc163d32`. Rule list: `[]` (Enabled: `True`). {{(pid=81449) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:28:00.161029 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-3452afd0-913d-4905-b5e2-6d2a42f70a28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:28:00.215055 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1159/2291] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 12:28:00 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:28:10.207960 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=25 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:28:23.430904 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5289b041-e5db-4a12-9d2a-0c0600c0ac10 None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:28:23.431363 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5289b041-e5db-4a12-9d2a-0c0600c0ac10 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:28:23.431463 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5289b041-e5db-4a12-9d2a-0c0600c0ac10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:28:23.454842 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-5289b041-e5db-4a12-9d2a-0c0600c0ac10 None None] MFA Rules not processed for user `89cc4de76c6a4305af9a9506840e13f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:28:23.456075 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-5289b041-e5db-4a12-9d2a-0c0600c0ac10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:28:23.509645 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1133/2292] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 12:28:23 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:28:26.587586 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a044db-177c-4075-9525-2425de578a63 None None] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:28:26.588194 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a044db-177c-4075-9525-2425de578a63 None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:28:26.588194 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a044db-177c-4075-9525-2425de578a63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:28:26.611719 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.auth.core [None req-98a044db-177c-4075-9525-2425de578a63 None None] MFA Rules not processed for user `20ca976222454c6c898d0e4b170b41fc`. Rule list: `[]` (Enabled: `True`). {{(pid=81449) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:28:26.613199 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-98a044db-177c-4075-9525-2425de578a63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:28:26.624345 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1160/2293] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 12:28:26 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:28:33.504031 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=25 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:28:36.604400 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=3 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:28:43.147368 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04db0d44-9957-4299-b9a4-f03bfb42b6ae None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:28:43.147842 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04db0d44-9957-4299-b9a4-f03bfb42b6ae None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:28:43.148000 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04db0d44-9957-4299-b9a4-f03bfb42b6ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:28:43.166769 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-04db0d44-9957-4299-b9a4-f03bfb42b6ae None None] MFA Rules not processed for user `4ead4be9666942f2b81cbb0f9ab69325`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:28:43.167960 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-04db0d44-9957-4299-b9a4-f03bfb42b6ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:28:43.174670 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1134/2294] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 12:28:43 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:28:53.160961 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=3 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:29:09.258131 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a044db-177c-4075-9525-2425de578a63 None None] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:29:09.268286 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a044db-177c-4075-9525-2425de578a63 None None] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:29:09.271875 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2b4bb9-181a-4513-ae3b-3bb6d0738b9d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfbb4d6809064f9cb9e06626ed7d6814', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '540a32ef88764e8c8d9201ae22f6ddaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:29:09.272471 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2b4bb9-181a-4513-ae3b-3bb6d0738b9d service neutron] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:29:09.272749 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2b4bb9-181a-4513-ae3b-3bb6d0738b9d service neutron] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:29:09.273044 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2b4bb9-181a-4513-ae3b-3bb6d0738b9d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:29:09.274730 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-2a2b4bb9-181a-4513-ae3b-3bb6d0738b9d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:29:09.288968 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2b4bb9-181a-4513-ae3b-3bb6d0738b9d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:29:09.290395 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2b4bb9-181a-4513-ae3b-3bb6d0738b9d service neutron] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:29:09.301203 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1161/2295] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 12:29:09 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 12:29:11.608081 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04db0d44-9957-4299-b9a4-f03bfb42b6ae None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:29:11.618438 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04db0d44-9957-4299-b9a4-f03bfb42b6ae None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:29:11.622025 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ec682b-c42a-48ef-a834-724f6ded3f31 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfbb4d6809064f9cb9e06626ed7d6814', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '540a32ef88764e8c8d9201ae22f6ddaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:29:11.622621 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ec682b-c42a-48ef-a834-724f6ded3f31 service neutron] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:29:11.622895 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ec682b-c42a-48ef-a834-724f6ded3f31 service neutron] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:29:11.623161 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ec682b-c42a-48ef-a834-724f6ded3f31 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:29:11.624952 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-76ec682b-c42a-48ef-a834-724f6ded3f31 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:29:11.638815 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ec682b-c42a-48ef-a834-724f6ded3f31 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:29:11.640478 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ec682b-c42a-48ef-a834-724f6ded3f31 service neutron] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:29:11.650996 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1135/2296] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 12:29:11 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 12:29:18.686684 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5a7360-6a50-4424-b9ee-1d3dc4b3c82d None None] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:29:18.687132 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5a7360-6a50-4424-b9ee-1d3dc4b3c82d None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:29:18.687280 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5a7360-6a50-4424-b9ee-1d3dc4b3c82d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:29:18.707702 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.auth.core [None req-bf5a7360-6a50-4424-b9ee-1d3dc4b3c82d None None] MFA Rules not processed for user `527f4dd6031546c785bc6360dacb60af`. Rule list: `[]` (Enabled: `True`). {{(pid=81449) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:29:18.709039 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-bf5a7360-6a50-4424-b9ee-1d3dc4b3c82d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:29:18.716394 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1162/2297] 10.222.0.15 () {58 vars in 1009 bytes} [Thu Sep 15 12:29:18 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:29:18.762374 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ec682b-c42a-48ef-a834-724f6ded3f31 service neutron] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:29:18.774184 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ec682b-c42a-48ef-a834-724f6ded3f31 service neutron] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:29:18.779108 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73601ee6-cd0e-40ec-bc13-1b4efa18dcca service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa0d3dc5b7540d59fdd59bbf64a04b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '540a32ef88764e8c8d9201ae22f6ddaf', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:29:18.779710 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73601ee6-cd0e-40ec-bc13-1b4efa18dcca service swift] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:29:18.780017 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73601ee6-cd0e-40ec-bc13-1b4efa18dcca service swift] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:29:18.780296 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73601ee6-cd0e-40ec-bc13-1b4efa18dcca service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:29:18.788652 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73601ee6-cd0e-40ec-bc13-1b4efa18dcca service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:29:18.790245 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73601ee6-cd0e-40ec-bc13-1b4efa18dcca service swift] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:29:18.798401 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1136/2298] 10.222.0.15 () {60 vars in 1384 bytes} [Thu Sep 15 12:29:18 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 15 12:29:24.019519 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5a7360-6a50-4424-b9ee-1d3dc4b3c82d None None] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:29:24.021474 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-bf5a7360-6a50-4424-b9ee-1d3dc4b3c82d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:29:24.042055 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5a7360-6a50-4424-b9ee-1d3dc4b3c82d None None] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:29:24.054227 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9ed301-937f-4c3c-995e-5b9fe0224087 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eface89e261840a2abb4206be26455c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '540a32ef88764e8c8d9201ae22f6ddaf', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:29:24.054934 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9ed301-937f-4c3c-995e-5b9fe0224087 service nova] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:29:24.055481 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9ed301-937f-4c3c-995e-5b9fe0224087 service nova] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:29:24.055843 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9ed301-937f-4c3c-995e-5b9fe0224087 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:29:24.057693 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-ad9ed301-937f-4c3c-995e-5b9fe0224087 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:29:24.071676 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9ed301-937f-4c3c-995e-5b9fe0224087 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:29:24.073559 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9ed301-937f-4c3c-995e-5b9fe0224087 service nova] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:29:24.085214 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1163/2299] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 12:29:24 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 66 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 12:29:24.877748 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73601ee6-cd0e-40ec-bc13-1b4efa18dcca service swift] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:29:24.880611 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-73601ee6-cd0e-40ec-bc13-1b4efa18dcca service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:29:24.897028 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73601ee6-cd0e-40ec-bc13-1b4efa18dcca service swift] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:29:24.904701 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbc328a-79ac-4d6d-aef2-7a5decdb9444 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eface89e261840a2abb4206be26455c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '540a32ef88764e8c8d9201ae22f6ddaf', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:29:24.906042 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbc328a-79ac-4d6d-aef2-7a5decdb9444 service nova] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:29:24.906491 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbc328a-79ac-4d6d-aef2-7a5decdb9444 service nova] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:29:24.906920 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbc328a-79ac-4d6d-aef2-7a5decdb9444 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:29:24.909224 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-efbc328a-79ac-4d6d-aef2-7a5decdb9444 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:29:24.926578 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efbc328a-79ac-4d6d-aef2-7a5decdb9444 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:29:24.928597 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efbc328a-79ac-4d6d-aef2-7a5decdb9444 service nova] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:29:24.954432 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1137/2300] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 12:29:24 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 77 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 12:29:33.547642 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9ed301-937f-4c3c-995e-5b9fe0224087 service nova] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:29:33.559650 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9ed301-937f-4c3c-995e-5b9fe0224087 service nova] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:29:33.564894 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f554a59f-4d04-4e5d-9c76-cba47e904a93 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f64a058bd5440cb5268f228aa6e392', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '540a32ef88764e8c8d9201ae22f6ddaf', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:29:33.566049 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f554a59f-4d04-4e5d-9c76-cba47e904a93 service placement] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:29:33.566340 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f554a59f-4d04-4e5d-9c76-cba47e904a93 service placement] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:29:33.566610 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f554a59f-4d04-4e5d-9c76-cba47e904a93 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:29:33.568354 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-f554a59f-4d04-4e5d-9c76-cba47e904a93 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:29:33.580394 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f554a59f-4d04-4e5d-9c76-cba47e904a93 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:29:33.582640 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f554a59f-4d04-4e5d-9c76-cba47e904a93 service placement] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:29:33.594393 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1164/2301] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 12:29:33 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 12:29:34.934818 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=18 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:29:36.309344 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895ee8e9-1829-471a-aaa9-48cbe74425fc None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:29:36.309895 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895ee8e9-1829-471a-aaa9-48cbe74425fc None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:29:36.309895 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895ee8e9-1829-471a-aaa9-48cbe74425fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:29:36.310143 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f554a59f-4d04-4e5d-9c76-cba47e904a93 service placement] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:29:36.311741 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-f554a59f-4d04-4e5d-9c76-cba47e904a93 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:29:36.331698 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-895ee8e9-1829-471a-aaa9-48cbe74425fc None None] MFA Rules not processed for user `527f4dd6031546c785bc6360dacb60af`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:29:36.333711 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-895ee8e9-1829-471a-aaa9-48cbe74425fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:29:36.341129 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1138/2302] 10.222.0.15 () {58 vars in 1009 bytes} [Thu Sep 15 12:29:36 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:29:36.353870 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895ee8e9-1829-471a-aaa9-48cbe74425fc None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:29:36.384985 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895ee8e9-1829-471a-aaa9-48cbe74425fc None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:29:36.384985 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352d426d-c87a-41c5-8650-8ddcaeecd259 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa0d3dc5b7540d59fdd59bbf64a04b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '540a32ef88764e8c8d9201ae22f6ddaf', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:29:36.384985 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352d426d-c87a-41c5-8650-8ddcaeecd259 service swift] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:29:36.384985 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352d426d-c87a-41c5-8650-8ddcaeecd259 service swift] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:29:36.385399 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352d426d-c87a-41c5-8650-8ddcaeecd259 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:29:36.388304 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352d426d-c87a-41c5-8650-8ddcaeecd259 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:29:36.389977 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352d426d-c87a-41c5-8650-8ddcaeecd259 service swift] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:29:36.399176 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1139/2303] 10.222.0.15 () {60 vars in 1384 bytes} [Thu Sep 15 12:29:36 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 15 12:29:36.413850 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f554a59f-4d04-4e5d-9c76-cba47e904a93 service placement] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:29:36.426200 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a92c37-ce91-4729-a3cd-cd2bdeac4b2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:29:36.426919 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a92c37-ce91-4729-a3cd-cd2bdeac4b2e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:29:36.427267 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a92c37-ce91-4729-a3cd-cd2bdeac4b2e admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:29:36.427603 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a92c37-ce91-4729-a3cd-cd2bdeac4b2e admin admin] PATH_INFO: `/v3/users/20ca976222454c6c898d0e4b170b41fc` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:29:36.429389 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a92c37-ce91-4729-a3cd-cd2bdeac4b2e admin admin] RBAC: Authorizing `identity:delete_user(user_id=20ca976222454c6c898d0e4b170b41fc)` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:29:36.431295 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a92c37-ce91-4729-a3cd-cd2bdeac4b2e admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:29:36.495847 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-13a92c37-ce91-4729-a3cd-cd2bdeac4b2e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '20ca976222454c6c898d0e4b170b41fc'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:29:36.506954 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-13a92c37-ce91-4729-a3cd-cd2bdeac4b2e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '20ca976222454c6c898d0e4b170b41fc'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:29:36.523961 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-13a92c37-ce91-4729-a3cd-cd2bdeac4b2e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '20ca976222454c6c898d0e4b170b41fc'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:29:36.525343 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-13a92c37-ce91-4729-a3cd-cd2bdeac4b2e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '20ca976222454c6c898d0e4b170b41fc'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:29:36.535687 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-13a92c37-ce91-4729-a3cd-cd2bdeac4b2e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '20ca976222454c6c898d0e4b170b41fc'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:29:36.543233 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1165/2304] 10.222.0.15 () {58 vars in 1224 bytes} [Thu Sep 15 12:29:36 2022] DELETE /identity/v3/users/20ca976222454c6c898d0e4b170b41fc => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 12:29:36.872115 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352d426d-c87a-41c5-8650-8ddcaeecd259 service swift] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:29:36.900357 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352d426d-c87a-41c5-8650-8ddcaeecd259 service swift] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:29:37.045916 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfded6cc-6875-461d-831f-b268df202355 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:29:37.046570 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfded6cc-6875-461d-831f-b268df202355 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:29:37.046908 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfded6cc-6875-461d-831f-b268df202355 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:29:37.047182 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfded6cc-6875-461d-831f-b268df202355 admin admin] PATH_INFO: `/v3/projects/22801aa5908b4b098931d2588433e9db` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:29:37.048617 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfded6cc-6875-461d-831f-b268df202355 admin admin] RBAC: Authorizing `identity:delete_project(project_id=22801aa5908b4b098931d2588433e9db)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:29:37.051387 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfded6cc-6875-461d-831f-b268df202355 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:29:37.178755 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-dfded6cc-6875-461d-831f-b268df202355 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '22801aa5908b4b098931d2588433e9db', 'request_id': 'req-dfded6cc-6875-461d-831f-b268df202355'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:29:37.209095 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-dfded6cc-6875-461d-831f-b268df202355 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '22801aa5908b4b098931d2588433e9db', 'request_id': 'req-dfded6cc-6875-461d-831f-b268df202355'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:29:37.225528 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-dfded6cc-6875-461d-831f-b268df202355 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '22801aa5908b4b098931d2588433e9db', 'request_id': 'req-dfded6cc-6875-461d-831f-b268df202355'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:29:37.244925 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-dfded6cc-6875-461d-831f-b268df202355 admin admin] The token cache is being invalidate because project 22801aa5908b4b098931d2588433e9db was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81448) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 12:29:37.245266 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-dfded6cc-6875-461d-831f-b268df202355 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:29:37.246004 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-dfded6cc-6875-461d-831f-b268df202355 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:29:37.251793 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1140/2305] 10.222.0.15 () {58 vars in 1233 bytes} [Thu Sep 15 12:29:36 2022] DELETE /identity/v3/projects/22801aa5908b4b098931d2588433e9db => generated 0 bytes in 376 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:29:37.253919 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7aab2f-ef00-48de-8482-d7460d44b2a6 None None] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:29:37.254213 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7aab2f-ef00-48de-8482-d7460d44b2a6 None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:29:37.254486 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7aab2f-ef00-48de-8482-d7460d44b2a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:29:37.263143 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=75,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:29:37.280542 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.auth.core [None req-2d7aab2f-ef00-48de-8482-d7460d44b2a6 None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81449) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:29:37.282498 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-2d7aab2f-ef00-48de-8482-d7460d44b2a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:29:37.377574 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1166/2306] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 12:29:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:29:37.388141 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3e4a5c-631a-4f85-a1e1-4a8e867d485f None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:29:37.388684 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3e4a5c-631a-4f85-a1e1-4a8e867d485f None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:29:37.389082 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce3e4a5c-631a-4f85-a1e1-4a8e867d485f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:29:37.419320 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-ce3e4a5c-631a-4f85-a1e1-4a8e867d485f None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:29:37.420907 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-ce3e4a5c-631a-4f85-a1e1-4a8e867d485f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:29:37.436133 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1141/2307] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 12:29:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:29:37.441056 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7aab2f-ef00-48de-8482-d7460d44b2a6 None None] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:29:37.462854 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7aab2f-ef00-48de-8482-d7460d44b2a6 None None] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:29:37.472577 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf1a6b8-3ea3-4843-b063-0fc44b237413 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:29:37.480274 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf1a6b8-3ea3-4843-b063-0fc44b237413 admin admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:29:37.481016 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf1a6b8-3ea3-4843-b063-0fc44b237413 admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:29:37.481586 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf1a6b8-3ea3-4843-b063-0fc44b237413 admin admin] PATH_INFO: `/v3/domains` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:29:37.484916 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf1a6b8-3ea3-4843-b063-0fc44b237413 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81449) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:29:37.485485 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf1a6b8-3ea3-4843-b063-0fc44b237413 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:29:37.487582 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf1a6b8-3ea3-4843-b063-0fc44b237413 admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:29:37.511115 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3e4a5c-631a-4f85-a1e1-4a8e867d485f None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:29:37.512307 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1167/2308] 10.222.0.15 () {58 vars in 1153 bytes} [Thu Sep 15 12:29:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:29:37.529129 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce3e4a5c-631a-4f85-a1e1-4a8e867d485f None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:29:37.535722 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d13fcf1-e5c1-4deb-9d44-550dd442c8f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:29:37.536463 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d13fcf1-e5c1-4deb-9d44-550dd442c8f1 admin admin] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:29:37.536969 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d13fcf1-e5c1-4deb-9d44-550dd442c8f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:29:37.537311 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d13fcf1-e5c1-4deb-9d44-550dd442c8f1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:29:37.538335 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d13fcf1-e5c1-4deb-9d44-550dd442c8f1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:29:37.540496 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d13fcf1-e5c1-4deb-9d44-550dd442c8f1 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:29:37.605976 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1142/2309] 10.222.0.15 () {60 vars in 1153 bytes} [Thu Sep 15 12:29:37 2022] POST /identity/v3/projects => generated 405 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 12:29:37.610697 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf1a6b8-3ea3-4843-b063-0fc44b237413 admin admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:29:37.623287 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf1a6b8-3ea3-4843-b063-0fc44b237413 admin admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:29:37.703798 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21fdb74b-8008-4cc3-913c-ef5781aa98bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:29:37.704544 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fdb74b-8008-4cc3-913c-ef5781aa98bc admin admin] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:29:37.704915 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fdb74b-8008-4cc3-913c-ef5781aa98bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:29:37.705256 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fdb74b-8008-4cc3-913c-ef5781aa98bc admin admin] PATH_INFO: `/v3/users` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:29:37.706504 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21fdb74b-8008-4cc3-913c-ef5781aa98bc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:29:37.710652 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21fdb74b-8008-4cc3-913c-ef5781aa98bc admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:29:37.736794 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1168/2310] 10.222.0.15 () {60 vars in 1144 bytes} [Thu Sep 15 12:29:37 2022] POST /identity/v3/users => generated 303 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 12:29:37.739808 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d13fcf1-e5c1-4deb-9d44-550dd442c8f1 admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:29:37.754826 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d13fcf1-e5c1-4deb-9d44-550dd442c8f1 admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:29:37.759937 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97dd2da-2b3a-494e-ad5f-6c9f9da7ff74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:29:37.760557 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97dd2da-2b3a-494e-ad5f-6c9f9da7ff74 admin admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:29:37.760859 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97dd2da-2b3a-494e-ad5f-6c9f9da7ff74 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:29:37.761131 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97dd2da-2b3a-494e-ad5f-6c9f9da7ff74 admin admin] PATH_INFO: `/v3/roles` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:29:37.761777 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97dd2da-2b3a-494e-ad5f-6c9f9da7ff74 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:29:37.763358 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97dd2da-2b3a-494e-ad5f-6c9f9da7ff74 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:29:37.774561 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1143/2311] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 12:29:37 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 15 12:29:37.778740 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21fdb74b-8008-4cc3-913c-ef5781aa98bc admin admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:29:37.790280 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21fdb74b-8008-4cc3-913c-ef5781aa98bc admin admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:29:37.797746 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f15a09-e0cc-4586-b1e6-3936f79aeee0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:29:37.798366 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f15a09-e0cc-4586-b1e6-3936f79aeee0 admin admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:29:37.798630 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f15a09-e0cc-4586-b1e6-3936f79aeee0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:29:37.798897 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f15a09-e0cc-4586-b1e6-3936f79aeee0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:29:37.799539 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f15a09-e0cc-4586-b1e6-3936f79aeee0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:29:37.801218 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f15a09-e0cc-4586-b1e6-3936f79aeee0 admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:29:37.813188 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1169/2312] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 12:29:37 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 12:29:37.817518 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97dd2da-2b3a-494e-ad5f-6c9f9da7ff74 admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:29:37.828951 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97dd2da-2b3a-494e-ad5f-6c9f9da7ff74 admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:29:37.834214 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c0b137-4e07-4826-bda1-0586530af96b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:29:37.834982 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c0b137-4e07-4826-bda1-0586530af96b admin admin] REQUEST_METHOD: `PUT` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:29:37.835334 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c0b137-4e07-4826-bda1-0586530af96b admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:29:37.835679 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c0b137-4e07-4826-bda1-0586530af96b admin admin] PATH_INFO: `/v3/projects/b5296631b7ec4bd98736f4bf46f03fc2/users/e03c90eb96624c7fbe2a9ecdffa1fadc/roles/0679d6628a25438d90dc4bdc0401eda2` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:29:37.853743 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c0b137-4e07-4826-bda1-0586530af96b admin admin] RBAC: Authorizing `identity:create_grant(project_id=b5296631b7ec4bd98736f4bf46f03fc2, user_id=e03c90eb96624c7fbe2a9ecdffa1fadc, role_id=0679d6628a25438d90dc4bdc0401eda2)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:29:37.856789 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c0b137-4e07-4826-bda1-0586530af96b admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:29:37.870497 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1144/2313] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 15 12:29:37 2022] PUT /identity/v3/projects/b5296631b7ec4bd98736f4bf46f03fc2/users/e03c90eb96624c7fbe2a9ecdffa1fadc/roles/0679d6628a25438d90dc4bdc0401eda2 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 12:29:37.883150 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74191e58-7bd7-4e8f-96dc-12aba3e9944d None None] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:29:37.883432 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74191e58-7bd7-4e8f-96dc-12aba3e9944d None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:29:37.883703 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74191e58-7bd7-4e8f-96dc-12aba3e9944d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:29:37.915885 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.auth.core [None req-74191e58-7bd7-4e8f-96dc-12aba3e9944d None None] MFA Rules not processed for user `e03c90eb96624c7fbe2a9ecdffa1fadc`. Rule list: `[]` (Enabled: `True`). {{(pid=81449) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:29:37.917121 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-74191e58-7bd7-4e8f-96dc-12aba3e9944d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:29:38.005026 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:29:38.010866 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1170/2314] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 12:29:37 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:29:38.015411 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c0b137-4e07-4826-bda1-0586530af96b admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:29:38.026136 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:29:38.095166 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c0b137-4e07-4826-bda1-0586530af96b admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:29:38.100843 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654b5d0f-7a6a-4fca-8287-b80b1487e089 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:29:38.101539 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654b5d0f-7a6a-4fca-8287-b80b1487e089 admin admin] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:29:38.101824 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654b5d0f-7a6a-4fca-8287-b80b1487e089 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:29:38.102100 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654b5d0f-7a6a-4fca-8287-b80b1487e089 admin admin] PATH_INFO: `/v3/projects` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:29:38.102940 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-654b5d0f-7a6a-4fca-8287-b80b1487e089 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:29:38.105092 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-654b5d0f-7a6a-4fca-8287-b80b1487e089 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:29:38.134707 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1145/2315] 10.222.0.15 () {60 vars in 1153 bytes} [Thu Sep 15 12:29:38 2022] POST /identity/v3/projects => generated 407 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 12:29:38.138858 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74191e58-7bd7-4e8f-96dc-12aba3e9944d None None] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:29:38.149993 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74191e58-7bd7-4e8f-96dc-12aba3e9944d None None] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:29:38.228815 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e148c82c-8417-4071-9e04-8f2f943a679b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:29:38.229445 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e148c82c-8417-4071-9e04-8f2f943a679b admin admin] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:29:38.229724 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e148c82c-8417-4071-9e04-8f2f943a679b admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:29:38.229994 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e148c82c-8417-4071-9e04-8f2f943a679b admin admin] PATH_INFO: `/v3/users` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:29:38.230677 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e148c82c-8417-4071-9e04-8f2f943a679b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:29:38.232543 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e148c82c-8417-4071-9e04-8f2f943a679b admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:29:38.284690 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1171/2316] 10.222.0.15 () {60 vars in 1144 bytes} [Thu Sep 15 12:29:38 2022] POST /identity/v3/users => generated 310 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 15 12:29:38.288352 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654b5d0f-7a6a-4fca-8287-b80b1487e089 admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:29:38.300188 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654b5d0f-7a6a-4fca-8287-b80b1487e089 admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:29:38.306736 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3bf4b3b-1158-4389-a950-65dfdde970df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:29:38.307586 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bf4b3b-1158-4389-a950-65dfdde970df admin admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:29:38.308092 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bf4b3b-1158-4389-a950-65dfdde970df admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:29:38.308567 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bf4b3b-1158-4389-a950-65dfdde970df admin admin] PATH_INFO: `/v3/roles` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:29:38.309514 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3bf4b3b-1158-4389-a950-65dfdde970df admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:29:38.311586 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3bf4b3b-1158-4389-a950-65dfdde970df admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:29:38.328923 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1146/2317] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 12:29:38 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 12:29:38.330100 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e148c82c-8417-4071-9e04-8f2f943a679b admin admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:29:38.343067 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e148c82c-8417-4071-9e04-8f2f943a679b admin admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:29:38.349079 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128fdee5-2e45-4956-b42e-4ed1f019cd14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:29:38.349850 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128fdee5-2e45-4956-b42e-4ed1f019cd14 admin admin] REQUEST_METHOD: `PUT` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:29:38.350199 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128fdee5-2e45-4956-b42e-4ed1f019cd14 admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:29:38.350545 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128fdee5-2e45-4956-b42e-4ed1f019cd14 admin admin] PATH_INFO: `/v3/projects/73f9b10bf2f1458f8ecb1603328d8e41/users/1ae0ad5f8b84407fa5252c99514d15be/roles/9c769b5c82884766b2c510b1f0f1364c` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:29:38.378185 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128fdee5-2e45-4956-b42e-4ed1f019cd14 admin admin] RBAC: Authorizing `identity:create_grant(project_id=73f9b10bf2f1458f8ecb1603328d8e41, user_id=1ae0ad5f8b84407fa5252c99514d15be, role_id=9c769b5c82884766b2c510b1f0f1364c)` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:29:38.381216 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128fdee5-2e45-4956-b42e-4ed1f019cd14 admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:29:38.402841 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1172/2318] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 15 12:29:38 2022] PUT /identity/v3/projects/73f9b10bf2f1458f8ecb1603328d8e41/users/1ae0ad5f8b84407fa5252c99514d15be/roles/9c769b5c82884766b2c510b1f0f1364c => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:29:38.406638 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3bf4b3b-1158-4389-a950-65dfdde970df admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:29:38.418873 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3bf4b3b-1158-4389-a950-65dfdde970df admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:29:38.500477 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ca861c-f274-43ec-b6b0-613e97091b68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:29:38.501109 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ca861c-f274-43ec-b6b0-613e97091b68 admin admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:29:38.501386 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ca861c-f274-43ec-b6b0-613e97091b68 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:29:38.501654 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ca861c-f274-43ec-b6b0-613e97091b68 admin admin] PATH_INFO: `/v3/roles` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:29:38.502302 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ca861c-f274-43ec-b6b0-613e97091b68 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:29:38.504523 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ca861c-f274-43ec-b6b0-613e97091b68 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:29:38.516640 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1147/2319] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 12:29:38 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 12:29:38.520428 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128fdee5-2e45-4956-b42e-4ed1f019cd14 admin admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:29:38.535295 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128fdee5-2e45-4956-b42e-4ed1f019cd14 admin admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:29:38.544546 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4efd526e-2e68-449a-9dda-bd2f3e289a7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:29:38.545253 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efd526e-2e68-449a-9dda-bd2f3e289a7c admin admin] REQUEST_METHOD: `PUT` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:29:38.545530 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efd526e-2e68-449a-9dda-bd2f3e289a7c admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:29:38.545798 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efd526e-2e68-449a-9dda-bd2f3e289a7c admin admin] PATH_INFO: `/v3/projects/73f9b10bf2f1458f8ecb1603328d8e41/users/1ae0ad5f8b84407fa5252c99514d15be/roles/9c769b5c82884766b2c510b1f0f1364c` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:29:38.547757 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4efd526e-2e68-449a-9dda-bd2f3e289a7c admin admin] RBAC: Authorizing `identity:create_grant(project_id=73f9b10bf2f1458f8ecb1603328d8e41, user_id=1ae0ad5f8b84407fa5252c99514d15be, role_id=9c769b5c82884766b2c510b1f0f1364c)` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:29:38.550986 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4efd526e-2e68-449a-9dda-bd2f3e289a7c admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:29:38.566961 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1173/2320] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 15 12:29:38 2022] PUT /identity/v3/projects/73f9b10bf2f1458f8ecb1603328d8e41/users/1ae0ad5f8b84407fa5252c99514d15be/roles/9c769b5c82884766b2c510b1f0f1364c => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 12:29:38.578574 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20167414-63b5-4640-8ea6-ff0bdd21d5ac None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:29:38.578882 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20167414-63b5-4640-8ea6-ff0bdd21d5ac None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:29:38.579136 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20167414-63b5-4640-8ea6-ff0bdd21d5ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:29:38.616161 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-20167414-63b5-4640-8ea6-ff0bdd21d5ac None None] MFA Rules not processed for user `1ae0ad5f8b84407fa5252c99514d15be`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:29:38.617325 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-20167414-63b5-4640-8ea6-ff0bdd21d5ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:29:38.702534 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1148/2321] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 12:29:38 2022] POST /identity/v3/auth/tokens => generated 3351 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:29:38.713657 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4efd526e-2e68-449a-9dda-bd2f3e289a7c admin admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:29:38.715779 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-4efd526e-2e68-449a-9dda-bd2f3e289a7c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:29:38.786056 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4efd526e-2e68-449a-9dda-bd2f3e289a7c admin admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:29:38.789837 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7151dc-b9e7-4128-9089-5c5ac3b693eb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8191d3ccb874529b32f8d0026db629d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '540a32ef88764e8c8d9201ae22f6ddaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:29:38.790545 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7151dc-b9e7-4128-9089-5c5ac3b693eb service cinder] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:29:38.790892 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7151dc-b9e7-4128-9089-5c5ac3b693eb service cinder] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:29:38.791229 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7151dc-b9e7-4128-9089-5c5ac3b693eb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:29:38.804240 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7151dc-b9e7-4128-9089-5c5ac3b693eb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:29:38.805805 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7151dc-b9e7-4128-9089-5c5ac3b693eb service cinder] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:29:38.816527 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1174/2322] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 12:29:38 2022] GET /identity/v3/auth/tokens => generated 3351 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 12:29:39.003685 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20167414-63b5-4640-8ea6-ff0bdd21d5ac None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:29:39.008361 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:29:39.014419 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20167414-63b5-4640-8ea6-ff0bdd21d5ac None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:29:39.020633 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0142c9d-4523-4329-b563-c9c56c128589 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8191d3ccb874529b32f8d0026db629d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '540a32ef88764e8c8d9201ae22f6ddaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:29:39.021802 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0142c9d-4523-4329-b563-c9c56c128589 service cinder] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:29:39.022169 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0142c9d-4523-4329-b563-c9c56c128589 service cinder] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:29:39.022562 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0142c9d-4523-4329-b563-c9c56c128589 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:29:39.090083 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0142c9d-4523-4329-b563-c9c56c128589 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:29:39.091754 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0142c9d-4523-4329-b563-c9c56c128589 service cinder] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:29:39.104206 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1149/2323] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 12:29:39 2022] GET /identity/v3/auth/tokens => generated 3282 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 12:29:39.369093 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7151dc-b9e7-4128-9089-5c5ac3b693eb service cinder] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:29:39.378746 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-ef7151dc-b9e7-4128-9089-5c5ac3b693eb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:29:39.488295 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:29:39.518513 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7151dc-b9e7-4128-9089-5c5ac3b693eb service cinder] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:29:39.526039 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab3629d-7194-491b-a482-d0fede9a3b2c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eface89e261840a2abb4206be26455c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '540a32ef88764e8c8d9201ae22f6ddaf', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:29:39.526663 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab3629d-7194-491b-a482-d0fede9a3b2c service nova] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:29:39.526951 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab3629d-7194-491b-a482-d0fede9a3b2c service nova] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:29:39.527591 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab3629d-7194-491b-a482-d0fede9a3b2c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:29:39.529103 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-aab3629d-7194-491b-a482-d0fede9a3b2c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:29:39.605764 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab3629d-7194-491b-a482-d0fede9a3b2c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:29:39.607434 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab3629d-7194-491b-a482-d0fede9a3b2c service nova] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:29:39.620965 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1175/2324] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 12:29:39 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 253 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 12:29:47.585491 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0142c9d-4523-4329-b563-c9c56c128589 service cinder] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:29:47.646934 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0142c9d-4523-4329-b563-c9c56c128589 service cinder] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:29:47.651362 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba4cd56-02b9-4620-b65e-4506f1b917af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:29:47.651980 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba4cd56-02b9-4620-b65e-4506f1b917af admin admin] REQUEST_METHOD: `DELETE` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:29:47.652253 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba4cd56-02b9-4620-b65e-4506f1b917af admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:29:47.652526 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba4cd56-02b9-4620-b65e-4506f1b917af admin admin] PATH_INFO: `/v3/users/e03c90eb96624c7fbe2a9ecdffa1fadc` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:29:47.653647 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba4cd56-02b9-4620-b65e-4506f1b917af admin admin] RBAC: Authorizing `identity:delete_user(user_id=e03c90eb96624c7fbe2a9ecdffa1fadc)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:29:47.655580 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba4cd56-02b9-4620-b65e-4506f1b917af admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:29:47.699777 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-0ba4cd56-02b9-4620-b65e-4506f1b917af admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e03c90eb96624c7fbe2a9ecdffa1fadc'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:29:47.711545 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-0ba4cd56-02b9-4620-b65e-4506f1b917af admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e03c90eb96624c7fbe2a9ecdffa1fadc'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:29:47.719009 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-0ba4cd56-02b9-4620-b65e-4506f1b917af admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e03c90eb96624c7fbe2a9ecdffa1fadc'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:29:47.719682 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-0ba4cd56-02b9-4620-b65e-4506f1b917af admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e03c90eb96624c7fbe2a9ecdffa1fadc'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:29:47.727093 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-0ba4cd56-02b9-4620-b65e-4506f1b917af admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e03c90eb96624c7fbe2a9ecdffa1fadc'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:29:47.733181 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1150/2325] 10.222.0.15 () {58 vars in 1224 bytes} [Thu Sep 15 12:29:47 2022] DELETE /identity/v3/users/e03c90eb96624c7fbe2a9ecdffa1fadc => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:29:47.738148 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609e71f4-f92e-4f45-b590-effa5a73a494 None None] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:29:47.738656 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609e71f4-f92e-4f45-b590-effa5a73a494 None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:29:47.738756 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609e71f4-f92e-4f45-b590-effa5a73a494 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:29:47.758341 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.auth.core [None req-609e71f4-f92e-4f45-b590-effa5a73a494 None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81449) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:29:47.759698 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-609e71f4-f92e-4f45-b590-effa5a73a494 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:29:47.831463 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1176/2326] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 12:29:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:29:47.841977 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba4cd56-02b9-4620-b65e-4506f1b917af admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:29:47.843428 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-0ba4cd56-02b9-4620-b65e-4506f1b917af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:29:47.865949 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:29:47.903150 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba4cd56-02b9-4620-b65e-4506f1b917af admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:29:47.906973 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8fb80e6-0b51-47d1-9136-6399e8638bc8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfbb4d6809064f9cb9e06626ed7d6814', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '540a32ef88764e8c8d9201ae22f6ddaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:29:47.907581 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fb80e6-0b51-47d1-9136-6399e8638bc8 service neutron] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:29:47.907861 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fb80e6-0b51-47d1-9136-6399e8638bc8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:29:47.908132 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fb80e6-0b51-47d1-9136-6399e8638bc8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:29:47.917813 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8fb80e6-0b51-47d1-9136-6399e8638bc8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:29:47.919230 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8fb80e6-0b51-47d1-9136-6399e8638bc8 service neutron] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:29:47.928725 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1151/2327] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 12:29:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 15 12:29:48.241664 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609e71f4-f92e-4f45-b590-effa5a73a494 None None] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:29:48.243199 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-609e71f4-f92e-4f45-b590-effa5a73a494 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:29:48.255395 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609e71f4-f92e-4f45-b590-effa5a73a494 None None] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:29:48.260235 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe9c05b-dd9c-4c42-ac26-eb8f7b6e5bdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:29:48.260943 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe9c05b-dd9c-4c42-ac26-eb8f7b6e5bdc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:29:48.261214 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe9c05b-dd9c-4c42-ac26-eb8f7b6e5bdc admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:29:48.261484 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fe9c05b-dd9c-4c42-ac26-eb8f7b6e5bdc admin admin] PATH_INFO: `/v3/projects/b5296631b7ec4bd98736f4bf46f03fc2` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:29:48.262666 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe9c05b-dd9c-4c42-ac26-eb8f7b6e5bdc admin admin] RBAC: Authorizing `identity:delete_project(project_id=b5296631b7ec4bd98736f4bf46f03fc2)` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:29:48.264513 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fe9c05b-dd9c-4c42-ac26-eb8f7b6e5bdc admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:29:48.318109 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-5fe9c05b-dd9c-4c42-ac26-eb8f7b6e5bdc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b5296631b7ec4bd98736f4bf46f03fc2', 'request_id': 'req-5fe9c05b-dd9c-4c42-ac26-eb8f7b6e5bdc'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:29:48.325963 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-5fe9c05b-dd9c-4c42-ac26-eb8f7b6e5bdc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b5296631b7ec4bd98736f4bf46f03fc2', 'request_id': 'req-5fe9c05b-dd9c-4c42-ac26-eb8f7b6e5bdc'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:29:48.332155 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-5fe9c05b-dd9c-4c42-ac26-eb8f7b6e5bdc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b5296631b7ec4bd98736f4bf46f03fc2', 'request_id': 'req-5fe9c05b-dd9c-4c42-ac26-eb8f7b6e5bdc'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:29:48.343959 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-5fe9c05b-dd9c-4c42-ac26-eb8f7b6e5bdc admin admin] The token cache is being invalidate because project b5296631b7ec4bd98736f4bf46f03fc2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81449) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 12:29:48.343959 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-5fe9c05b-dd9c-4c42-ac26-eb8f7b6e5bdc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:29:48.344666 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-5fe9c05b-dd9c-4c42-ac26-eb8f7b6e5bdc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:29:48.347090 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1177/2328] 10.222.0.15 () {58 vars in 1233 bytes} [Thu Sep 15 12:29:48 2022] DELETE /identity/v3/projects/b5296631b7ec4bd98736f4bf46f03fc2 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:29:48.350903 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8fb80e6-0b51-47d1-9136-6399e8638bc8 service neutron] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:29:48.360733 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8fb80e6-0b51-47d1-9136-6399e8638bc8 service neutron] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:29:48.417229 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c052856-dd53-485f-a574-8b29a3fd9b99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:29:48.417855 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c052856-dd53-485f-a574-8b29a3fd9b99 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:29:48.418131 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c052856-dd53-485f-a574-8b29a3fd9b99 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:29:48.418398 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c052856-dd53-485f-a574-8b29a3fd9b99 admin admin] PATH_INFO: `/v3/users/1ae0ad5f8b84407fa5252c99514d15be` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:29:48.419484 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c052856-dd53-485f-a574-8b29a3fd9b99 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1ae0ad5f8b84407fa5252c99514d15be)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:29:48.421432 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c052856-dd53-485f-a574-8b29a3fd9b99 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:29:48.467944 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-3c052856-dd53-485f-a574-8b29a3fd9b99 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1ae0ad5f8b84407fa5252c99514d15be'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:29:48.480543 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-3c052856-dd53-485f-a574-8b29a3fd9b99 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1ae0ad5f8b84407fa5252c99514d15be'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:29:48.488298 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-3c052856-dd53-485f-a574-8b29a3fd9b99 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1ae0ad5f8b84407fa5252c99514d15be'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:29:48.488949 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-3c052856-dd53-485f-a574-8b29a3fd9b99 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1ae0ad5f8b84407fa5252c99514d15be'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:29:48.495864 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-3c052856-dd53-485f-a574-8b29a3fd9b99 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1ae0ad5f8b84407fa5252c99514d15be'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:29:48.501719 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1152/2329] 10.222.0.15 () {58 vars in 1224 bytes} [Thu Sep 15 12:29:48 2022] DELETE /identity/v3/users/1ae0ad5f8b84407fa5252c99514d15be => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:29:48.763779 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe9c05b-dd9c-4c42-ac26-eb8f7b6e5bdc admin admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:29:48.765267 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-5fe9c05b-dd9c-4c42-ac26-eb8f7b6e5bdc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:29:48.779725 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fe9c05b-dd9c-4c42-ac26-eb8f7b6e5bdc admin admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:29:48.789845 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:29:48.848056 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd0247c-c68d-4a58-8986-3e3d0135ab32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:29:48.848779 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd0247c-c68d-4a58-8986-3e3d0135ab32 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:29:48.849127 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd0247c-c68d-4a58-8986-3e3d0135ab32 admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:29:48.849468 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd0247c-c68d-4a58-8986-3e3d0135ab32 admin admin] PATH_INFO: `/v3/projects/73f9b10bf2f1458f8ecb1603328d8e41` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:29:48.850644 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd0247c-c68d-4a58-8986-3e3d0135ab32 admin admin] RBAC: Authorizing `identity:delete_project(project_id=73f9b10bf2f1458f8ecb1603328d8e41)` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:29:48.852677 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd0247c-c68d-4a58-8986-3e3d0135ab32 admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:29:48.902416 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-0cd0247c-c68d-4a58-8986-3e3d0135ab32 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '73f9b10bf2f1458f8ecb1603328d8e41', 'request_id': 'req-0cd0247c-c68d-4a58-8986-3e3d0135ab32'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:29:48.910772 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-0cd0247c-c68d-4a58-8986-3e3d0135ab32 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '73f9b10bf2f1458f8ecb1603328d8e41', 'request_id': 'req-0cd0247c-c68d-4a58-8986-3e3d0135ab32'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:29:48.916391 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-0cd0247c-c68d-4a58-8986-3e3d0135ab32 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '73f9b10bf2f1458f8ecb1603328d8e41', 'request_id': 'req-0cd0247c-c68d-4a58-8986-3e3d0135ab32'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:29:48.927189 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-0cd0247c-c68d-4a58-8986-3e3d0135ab32 admin admin] The token cache is being invalidate because project 73f9b10bf2f1458f8ecb1603328d8e41 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81449) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 12:29:48.927482 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-0cd0247c-c68d-4a58-8986-3e3d0135ab32 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:29:48.928140 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-0cd0247c-c68d-4a58-8986-3e3d0135ab32 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:29:48.929632 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1178/2330] 10.222.0.15 () {58 vars in 1233 bytes} [Thu Sep 15 12:29:48 2022] DELETE /identity/v3/projects/73f9b10bf2f1458f8ecb1603328d8e41 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:29:48.934671 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7572d844-9488-4a42-9d5c-0948ea43a75b None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:29:48.935158 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7572d844-9488-4a42-9d5c-0948ea43a75b None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:29:48.935275 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7572d844-9488-4a42-9d5c-0948ea43a75b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:29:48.955824 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-7572d844-9488-4a42-9d5c-0948ea43a75b None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:29:48.957047 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-7572d844-9488-4a42-9d5c-0948ea43a75b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:29:48.994792 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:29:49.017265 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1153/2331] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 12:29:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:29:49.025706 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9408a803-3bd2-4d8f-a7c2-b6d400eee02d None None] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:29:49.025992 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9408a803-3bd2-4d8f-a7c2-b6d400eee02d None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:29:49.026266 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9408a803-3bd2-4d8f-a7c2-b6d400eee02d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:29:49.046178 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.auth.core [None req-9408a803-3bd2-4d8f-a7c2-b6d400eee02d None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81449) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:29:49.047391 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-9408a803-3bd2-4d8f-a7c2-b6d400eee02d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:29:49.055592 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1179/2332] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 12:29:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:29:49.059884 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7572d844-9488-4a42-9d5c-0948ea43a75b None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:29:49.069929 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7572d844-9488-4a42-9d5c-0948ea43a75b None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:29:49.074600 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e843ec0a-6c6c-46f8-979b-4c9896a56491 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:29:49.075196 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e843ec0a-6c6c-46f8-979b-4c9896a56491 admin admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:29:49.075471 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e843ec0a-6c6c-46f8-979b-4c9896a56491 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:29:49.075734 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e843ec0a-6c6c-46f8-979b-4c9896a56491 admin admin] PATH_INFO: `/v3/domains` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:29:49.076369 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e843ec0a-6c6c-46f8-979b-4c9896a56491 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:29:49.076665 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e843ec0a-6c6c-46f8-979b-4c9896a56491 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:29:49.078270 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e843ec0a-6c6c-46f8-979b-4c9896a56491 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:29:49.090535 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1154/2333] 10.222.0.15 () {58 vars in 1153 bytes} [Thu Sep 15 12:29:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:29:49.094977 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9408a803-3bd2-4d8f-a7c2-b6d400eee02d None None] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:29:49.104598 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9408a803-3bd2-4d8f-a7c2-b6d400eee02d None None] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:29:49.109235 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50676d1f-ed33-4269-9abe-32338f376a0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:29:49.109824 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50676d1f-ed33-4269-9abe-32338f376a0b admin admin] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:29:49.110124 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50676d1f-ed33-4269-9abe-32338f376a0b admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:29:49.110393 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50676d1f-ed33-4269-9abe-32338f376a0b admin admin] PATH_INFO: `/v3/projects` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:29:49.111007 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50676d1f-ed33-4269-9abe-32338f376a0b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:29:49.112974 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50676d1f-ed33-4269-9abe-32338f376a0b admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:29:49.141969 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1180/2334] 10.222.0.15 () {60 vars in 1153 bytes} [Thu Sep 15 12:29:49 2022] POST /identity/v3/projects => generated 373 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 12:29:49.146610 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e843ec0a-6c6c-46f8-979b-4c9896a56491 admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:29:49.156199 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e843ec0a-6c6c-46f8-979b-4c9896a56491 admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:29:49.212459 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea11665-a66b-4fc7-956f-e04695a13e93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:29:49.213366 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea11665-a66b-4fc7-956f-e04695a13e93 admin admin] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:29:49.213642 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea11665-a66b-4fc7-956f-e04695a13e93 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:29:49.213964 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea11665-a66b-4fc7-956f-e04695a13e93 admin admin] PATH_INFO: `/v3/users` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:29:49.214520 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea11665-a66b-4fc7-956f-e04695a13e93 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:29:49.216449 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea11665-a66b-4fc7-956f-e04695a13e93 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:29:49.239428 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1155/2335] 10.222.0.15 () {60 vars in 1144 bytes} [Thu Sep 15 12:29:49 2022] POST /identity/v3/users => generated 287 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 12:29:49.243677 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50676d1f-ed33-4269-9abe-32338f376a0b admin admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:29:49.253846 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50676d1f-ed33-4269-9abe-32338f376a0b admin admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:29:49.258466 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ec4d7b-6a03-441e-ac19-9288fb3dd91f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:29:49.259081 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ec4d7b-6a03-441e-ac19-9288fb3dd91f admin admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:29:49.259360 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ec4d7b-6a03-441e-ac19-9288fb3dd91f admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:29:49.259631 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ec4d7b-6a03-441e-ac19-9288fb3dd91f admin admin] PATH_INFO: `/v3/roles` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:29:49.260270 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ec4d7b-6a03-441e-ac19-9288fb3dd91f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:29:49.262005 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ec4d7b-6a03-441e-ac19-9288fb3dd91f admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:29:49.275380 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1181/2336] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 12:29:49 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 12:29:49.276574 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea11665-a66b-4fc7-956f-e04695a13e93 admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:29:49.287345 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea11665-a66b-4fc7-956f-e04695a13e93 admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:29:49.291945 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5649d1ac-ae81-4675-8b17-e1f118088b8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:29:49.292546 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5649d1ac-ae81-4675-8b17-e1f118088b8d admin admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:29:49.292862 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5649d1ac-ae81-4675-8b17-e1f118088b8d admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:29:49.293133 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5649d1ac-ae81-4675-8b17-e1f118088b8d admin admin] PATH_INFO: `/v3/roles` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:29:49.293791 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5649d1ac-ae81-4675-8b17-e1f118088b8d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:29:49.295497 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5649d1ac-ae81-4675-8b17-e1f118088b8d admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:29:49.306527 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1156/2337] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 12:29:49 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 12:29:49.310824 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ec4d7b-6a03-441e-ac19-9288fb3dd91f admin admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:29:49.321296 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ec4d7b-6a03-441e-ac19-9288fb3dd91f admin admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:29:49.326031 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12564802-a00f-4e08-a283-40e27d374c75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:29:49.326703 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12564802-a00f-4e08-a283-40e27d374c75 admin admin] REQUEST_METHOD: `PUT` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:29:49.326986 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12564802-a00f-4e08-a283-40e27d374c75 admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:29:49.327261 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12564802-a00f-4e08-a283-40e27d374c75 admin admin] PATH_INFO: `/v3/projects/628f9d8efc5b445bb6cd0aadbdb77377/users/def9d06da082452fb8b5980ad194053d/roles/0679d6628a25438d90dc4bdc0401eda2` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:29:49.343678 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12564802-a00f-4e08-a283-40e27d374c75 admin admin] RBAC: Authorizing `identity:create_grant(project_id=628f9d8efc5b445bb6cd0aadbdb77377, user_id=def9d06da082452fb8b5980ad194053d, role_id=0679d6628a25438d90dc4bdc0401eda2)` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:29:49.346396 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12564802-a00f-4e08-a283-40e27d374c75 admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:29:49.360259 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1182/2338] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 15 12:29:49 2022] PUT /identity/v3/projects/628f9d8efc5b445bb6cd0aadbdb77377/users/def9d06da082452fb8b5980ad194053d/roles/0679d6628a25438d90dc4bdc0401eda2 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:29:49.375011 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53ecabb-cd38-4d89-a383-78a7bda1a931 None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:29:49.375291 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53ecabb-cd38-4d89-a383-78a7bda1a931 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:29:49.375556 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53ecabb-cd38-4d89-a383-78a7bda1a931 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:29:49.403734 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-e53ecabb-cd38-4d89-a383-78a7bda1a931 None None] MFA Rules not processed for user `def9d06da082452fb8b5980ad194053d`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:29:49.404967 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-e53ecabb-cd38-4d89-a383-78a7bda1a931 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:29:49.466911 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1157/2339] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 12:29:49 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:29:49.471303 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12564802-a00f-4e08-a283-40e27d374c75 admin admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:29:49.512221 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:29:49.532643 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12564802-a00f-4e08-a283-40e27d374c75 admin admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:29:49.537746 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3337357-fc97-4d77-b4b9-cb823963bdd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:29:49.538345 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3337357-fc97-4d77-b4b9-cb823963bdd4 admin admin] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:29:49.538620 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3337357-fc97-4d77-b4b9-cb823963bdd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:29:49.538888 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3337357-fc97-4d77-b4b9-cb823963bdd4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:29:49.539503 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3337357-fc97-4d77-b4b9-cb823963bdd4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:29:49.541430 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3337357-fc97-4d77-b4b9-cb823963bdd4 admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:29:49.567091 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1183/2340] 10.222.0.15 () {60 vars in 1153 bytes} [Thu Sep 15 12:29:49 2022] POST /identity/v3/projects => generated 375 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 12:29:49.571234 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53ecabb-cd38-4d89-a383-78a7bda1a931 None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:29:49.583987 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53ecabb-cd38-4d89-a383-78a7bda1a931 None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:29:49.616386 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:29:49.639178 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4013140-bfde-49fe-b04b-0f597e6edf8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:29:49.639787 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4013140-bfde-49fe-b04b-0f597e6edf8d admin admin] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:29:49.640067 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4013140-bfde-49fe-b04b-0f597e6edf8d admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:29:49.640342 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4013140-bfde-49fe-b04b-0f597e6edf8d admin admin] PATH_INFO: `/v3/users` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:29:49.640994 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4013140-bfde-49fe-b04b-0f597e6edf8d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:29:49.643440 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4013140-bfde-49fe-b04b-0f597e6edf8d admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:29:49.664295 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1158/2341] 10.222.0.15 () {60 vars in 1144 bytes} [Thu Sep 15 12:29:49 2022] POST /identity/v3/users => generated 294 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 12:29:49.668313 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3337357-fc97-4d77-b4b9-cb823963bdd4 admin admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:29:49.678554 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3337357-fc97-4d77-b4b9-cb823963bdd4 admin admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:29:49.683188 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5eba87-8e1e-41cb-8aa0-9b8cb824318d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:29:49.683797 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5eba87-8e1e-41cb-8aa0-9b8cb824318d admin admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:29:49.684074 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5eba87-8e1e-41cb-8aa0-9b8cb824318d admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:29:49.684346 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5eba87-8e1e-41cb-8aa0-9b8cb824318d admin admin] PATH_INFO: `/v3/roles` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:29:49.685008 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e5eba87-8e1e-41cb-8aa0-9b8cb824318d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:29:49.686592 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e5eba87-8e1e-41cb-8aa0-9b8cb824318d admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:29:49.697050 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1184/2342] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 12:29:49 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 12:29:49.701300 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4013140-bfde-49fe-b04b-0f597e6edf8d admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:29:49.711497 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4013140-bfde-49fe-b04b-0f597e6edf8d admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:29:49.716327 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb54a57b-8e06-49b4-bddf-e7132d129c04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:29:49.717009 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb54a57b-8e06-49b4-bddf-e7132d129c04 admin admin] REQUEST_METHOD: `PUT` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:29:49.717291 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb54a57b-8e06-49b4-bddf-e7132d129c04 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:29:49.717566 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb54a57b-8e06-49b4-bddf-e7132d129c04 admin admin] PATH_INFO: `/v3/projects/fd7fb948652341ebbc789ef959ffa085/users/aead9d71564f4871a5302614aa05bfde/roles/9c769b5c82884766b2c510b1f0f1364c` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:29:49.732430 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb54a57b-8e06-49b4-bddf-e7132d129c04 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fd7fb948652341ebbc789ef959ffa085, user_id=aead9d71564f4871a5302614aa05bfde, role_id=9c769b5c82884766b2c510b1f0f1364c)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:29:49.735112 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb54a57b-8e06-49b4-bddf-e7132d129c04 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:29:49.747688 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1159/2343] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 15 12:29:49 2022] PUT /identity/v3/projects/fd7fb948652341ebbc789ef959ffa085/users/aead9d71564f4871a5302614aa05bfde/roles/9c769b5c82884766b2c510b1f0f1364c => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:29:49.751524 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5eba87-8e1e-41cb-8aa0-9b8cb824318d admin admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:29:49.761274 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5eba87-8e1e-41cb-8aa0-9b8cb824318d admin admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:29:49.816876 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2fd0f7-3c82-4cee-8b43-e3247d7f61a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:29:49.817489 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2fd0f7-3c82-4cee-8b43-e3247d7f61a7 admin admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:29:49.817767 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2fd0f7-3c82-4cee-8b43-e3247d7f61a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:29:49.818037 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2fd0f7-3c82-4cee-8b43-e3247d7f61a7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:29:49.818673 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2fd0f7-3c82-4cee-8b43-e3247d7f61a7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:29:49.820251 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2fd0f7-3c82-4cee-8b43-e3247d7f61a7 admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:29:49.829981 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1185/2344] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 12:29:49 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 12:29:49.834635 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb54a57b-8e06-49b4-bddf-e7132d129c04 admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:29:49.846705 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb54a57b-8e06-49b4-bddf-e7132d129c04 admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:29:49.851473 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3db0960-e630-4f66-99f6-fbb1d90b9527 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:29:49.852293 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3db0960-e630-4f66-99f6-fbb1d90b9527 admin admin] REQUEST_METHOD: `PUT` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:29:49.852658 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3db0960-e630-4f66-99f6-fbb1d90b9527 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:29:49.852963 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3db0960-e630-4f66-99f6-fbb1d90b9527 admin admin] PATH_INFO: `/v3/projects/fd7fb948652341ebbc789ef959ffa085/users/aead9d71564f4871a5302614aa05bfde/roles/9c769b5c82884766b2c510b1f0f1364c` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:29:49.855642 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3db0960-e630-4f66-99f6-fbb1d90b9527 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fd7fb948652341ebbc789ef959ffa085, user_id=aead9d71564f4871a5302614aa05bfde, role_id=9c769b5c82884766b2c510b1f0f1364c)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:29:49.858422 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3db0960-e630-4f66-99f6-fbb1d90b9527 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:29:49.869946 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1160/2345] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 15 12:29:49 2022] PUT /identity/v3/projects/fd7fb948652341ebbc789ef959ffa085/users/aead9d71564f4871a5302614aa05bfde/roles/9c769b5c82884766b2c510b1f0f1364c => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:29:49.881603 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4c4cd5-86c1-42d8-b33f-cee871fa3b6c None None] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:29:49.882122 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4c4cd5-86c1-42d8-b33f-cee871fa3b6c None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:29:49.882497 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4c4cd5-86c1-42d8-b33f-cee871fa3b6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:29:49.921491 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.auth.core [None req-4e4c4cd5-86c1-42d8-b33f-cee871fa3b6c None None] MFA Rules not processed for user `aead9d71564f4871a5302614aa05bfde`. Rule list: `[]` (Enabled: `True`). {{(pid=81449) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:29:49.922976 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-4e4c4cd5-86c1-42d8-b33f-cee871fa3b6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:29:50.007613 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1186/2346] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 12:29:49 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:29:50.020141 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3db0960-e630-4f66-99f6-fbb1d90b9527 admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:29:50.021995 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-b3db0960-e630-4f66-99f6-fbb1d90b9527 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:29:50.088670 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3db0960-e630-4f66-99f6-fbb1d90b9527 admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:29:50.093001 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8bd0f64-777e-4a78-81a9-734e3d6ed05c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8191d3ccb874529b32f8d0026db629d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '540a32ef88764e8c8d9201ae22f6ddaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:29:50.093617 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bd0f64-777e-4a78-81a9-734e3d6ed05c service cinder] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:29:50.093893 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bd0f64-777e-4a78-81a9-734e3d6ed05c service cinder] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:29:50.094158 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bd0f64-777e-4a78-81a9-734e3d6ed05c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:29:50.153581 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8bd0f64-777e-4a78-81a9-734e3d6ed05c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:29:50.155479 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8bd0f64-777e-4a78-81a9-734e3d6ed05c service cinder] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:29:50.166439 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1161/2347] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 12:29:50 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 12:29:50.199387 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4c4cd5-86c1-42d8-b33f-cee871fa3b6c None None] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:29:50.201164 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-4e4c4cd5-86c1-42d8-b33f-cee871fa3b6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:29:50.243386 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:29:50.261706 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4c4cd5-86c1-42d8-b33f-cee871fa3b6c None None] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:29:50.265258 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5e3e80-9a85-47d9-bbbb-80b3fc00ccd7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfbb4d6809064f9cb9e06626ed7d6814', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '540a32ef88764e8c8d9201ae22f6ddaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:29:50.265860 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5e3e80-9a85-47d9-bbbb-80b3fc00ccd7 service neutron] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:29:50.266142 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5e3e80-9a85-47d9-bbbb-80b3fc00ccd7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:29:50.266410 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5e3e80-9a85-47d9-bbbb-80b3fc00ccd7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:29:50.268329 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-1c5e3e80-9a85-47d9-bbbb-80b3fc00ccd7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:29:50.277962 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5e3e80-9a85-47d9-bbbb-80b3fc00ccd7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:29:50.279390 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5e3e80-9a85-47d9-bbbb-80b3fc00ccd7 service neutron] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:29:50.288063 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1187/2348] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 12:29:50 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 12:29:51.661147 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a439442c-c02c-4310-9eca-d86a1822d9d2 None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:29:51.661649 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a439442c-c02c-4310-9eca-d86a1822d9d2 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:29:51.661733 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a439442c-c02c-4310-9eca-d86a1822d9d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:29:51.686759 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-a439442c-c02c-4310-9eca-d86a1822d9d2 None None] MFA Rules not processed for user `527f4dd6031546c785bc6360dacb60af`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:29:51.688251 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-a439442c-c02c-4310-9eca-d86a1822d9d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:29:51.745157 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:29:51.764685 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1162/2349] 10.222.0.15 () {58 vars in 967 bytes} [Thu Sep 15 12:29:51 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:29:51.777463 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5e3e80-9a85-47d9-bbbb-80b3fc00ccd7 service neutron] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:29:51.779017 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-1c5e3e80-9a85-47d9-bbbb-80b3fc00ccd7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:29:51.850815 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5e3e80-9a85-47d9-bbbb-80b3fc00ccd7 service neutron] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:29:51.857122 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22855bb3-6133-4537-bdf3-671cec59c02d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa0d3dc5b7540d59fdd59bbf64a04b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '540a32ef88764e8c8d9201ae22f6ddaf', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:29:51.857810 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22855bb3-6133-4537-bdf3-671cec59c02d service swift] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:29:51.858086 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22855bb3-6133-4537-bdf3-671cec59c02d service swift] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:29:51.858378 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22855bb3-6133-4537-bdf3-671cec59c02d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:29:51.867512 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22855bb3-6133-4537-bdf3-671cec59c02d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:29:51.869248 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22855bb3-6133-4537-bdf3-671cec59c02d service swift] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:29:51.877530 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1188/2350] 10.222.0.15 () {60 vars in 1384 bytes} [Thu Sep 15 12:29:51 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 15 12:29:52.786022 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a439442c-c02c-4310-9eca-d86a1822d9d2 None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:29:52.859672 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a439442c-c02c-4310-9eca-d86a1822d9d2 None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:29:52.864591 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316e5e1f-6acb-4a5f-81b5-d214a0fde9b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:29:52.865266 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316e5e1f-6acb-4a5f-81b5-d214a0fde9b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:29:52.865571 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316e5e1f-6acb-4a5f-81b5-d214a0fde9b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:29:52.865841 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316e5e1f-6acb-4a5f-81b5-d214a0fde9b7 admin admin] PATH_INFO: `/v3/users/def9d06da082452fb8b5980ad194053d` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:29:52.867113 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316e5e1f-6acb-4a5f-81b5-d214a0fde9b7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=def9d06da082452fb8b5980ad194053d)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:29:52.868960 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-316e5e1f-6acb-4a5f-81b5-d214a0fde9b7 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:29:52.915545 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-316e5e1f-6acb-4a5f-81b5-d214a0fde9b7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'def9d06da082452fb8b5980ad194053d'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:29:52.927922 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-316e5e1f-6acb-4a5f-81b5-d214a0fde9b7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'def9d06da082452fb8b5980ad194053d'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:29:52.935775 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-316e5e1f-6acb-4a5f-81b5-d214a0fde9b7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'def9d06da082452fb8b5980ad194053d'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:29:52.936427 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-316e5e1f-6acb-4a5f-81b5-d214a0fde9b7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'def9d06da082452fb8b5980ad194053d'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:29:52.944378 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-316e5e1f-6acb-4a5f-81b5-d214a0fde9b7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'def9d06da082452fb8b5980ad194053d'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:29:52.950722 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1163/2351] 10.222.0.15 () {58 vars in 1224 bytes} [Thu Sep 15 12:29:52 2022] DELETE /identity/v3/users/def9d06da082452fb8b5980ad194053d => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:29:52.955179 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5043854d-a910-4994-a617-e9ef68e21889 None None] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:29:52.955636 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5043854d-a910-4994-a617-e9ef68e21889 None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:29:52.955992 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5043854d-a910-4994-a617-e9ef68e21889 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:29:52.976509 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.auth.core [None req-5043854d-a910-4994-a617-e9ef68e21889 None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81449) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:29:52.978175 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-5043854d-a910-4994-a617-e9ef68e21889 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:29:53.038126 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1189/2352] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 12:29:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:29:53.047686 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316e5e1f-6acb-4a5f-81b5-d214a0fde9b7 admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:29:53.049139 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-316e5e1f-6acb-4a5f-81b5-d214a0fde9b7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:29:53.106931 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316e5e1f-6acb-4a5f-81b5-d214a0fde9b7 admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:29:53.113012 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589ef101-2915-455e-a2f1-11bb772c5805 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfbb4d6809064f9cb9e06626ed7d6814', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '540a32ef88764e8c8d9201ae22f6ddaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:29:53.113575 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589ef101-2915-455e-a2f1-11bb772c5805 service neutron] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:29:53.113859 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589ef101-2915-455e-a2f1-11bb772c5805 service neutron] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:29:53.114129 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589ef101-2915-455e-a2f1-11bb772c5805 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:29:53.123945 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589ef101-2915-455e-a2f1-11bb772c5805 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:29:53.125516 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589ef101-2915-455e-a2f1-11bb772c5805 service neutron] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:29:53.135502 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1164/2353] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 12:29:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 12:29:53.648329 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5043854d-a910-4994-a617-e9ef68e21889 None None] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:29:53.649773 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-5043854d-a910-4994-a617-e9ef68e21889 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:29:53.664098 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5043854d-a910-4994-a617-e9ef68e21889 None None] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:29:53.669103 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af384ee-4e28-44dc-9025-4559d3e81695 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:29:53.669767 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af384ee-4e28-44dc-9025-4559d3e81695 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:29:53.670397 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af384ee-4e28-44dc-9025-4559d3e81695 admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:29:53.670512 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af384ee-4e28-44dc-9025-4559d3e81695 admin admin] PATH_INFO: `/v3/projects/628f9d8efc5b445bb6cd0aadbdb77377` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:29:53.671780 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af384ee-4e28-44dc-9025-4559d3e81695 admin admin] RBAC: Authorizing `identity:delete_project(project_id=628f9d8efc5b445bb6cd0aadbdb77377)` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:29:53.674142 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af384ee-4e28-44dc-9025-4559d3e81695 admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:29:53.720884 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-7af384ee-4e28-44dc-9025-4559d3e81695 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '628f9d8efc5b445bb6cd0aadbdb77377', 'request_id': 'req-7af384ee-4e28-44dc-9025-4559d3e81695'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:29:53.728004 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-7af384ee-4e28-44dc-9025-4559d3e81695 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '628f9d8efc5b445bb6cd0aadbdb77377', 'request_id': 'req-7af384ee-4e28-44dc-9025-4559d3e81695'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:29:53.732730 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:29:53.733967 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-7af384ee-4e28-44dc-9025-4559d3e81695 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '628f9d8efc5b445bb6cd0aadbdb77377', 'request_id': 'req-7af384ee-4e28-44dc-9025-4559d3e81695'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:29:53.746541 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-7af384ee-4e28-44dc-9025-4559d3e81695 admin admin] The token cache is being invalidate because project 628f9d8efc5b445bb6cd0aadbdb77377 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81449) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 12:29:53.746854 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-7af384ee-4e28-44dc-9025-4559d3e81695 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:29:53.747488 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-7af384ee-4e28-44dc-9025-4559d3e81695 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:29:53.748972 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1190/2354] 10.222.0.15 () {58 vars in 1233 bytes} [Thu Sep 15 12:29:53 2022] DELETE /identity/v3/projects/628f9d8efc5b445bb6cd0aadbdb77377 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:29:53.753472 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589ef101-2915-455e-a2f1-11bb772c5805 service neutron] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:29:53.762783 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589ef101-2915-455e-a2f1-11bb772c5805 service neutron] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:29:53.770932 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:29:53.819582 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2ce223-3d4a-4692-a704-ae0d68c1e718 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:29:53.820244 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2ce223-3d4a-4692-a704-ae0d68c1e718 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:29:53.820477 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2ce223-3d4a-4692-a704-ae0d68c1e718 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:29:53.820749 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2ce223-3d4a-4692-a704-ae0d68c1e718 admin admin] PATH_INFO: `/v3/users/aead9d71564f4871a5302614aa05bfde` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:29:53.821886 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2ce223-3d4a-4692-a704-ae0d68c1e718 admin admin] RBAC: Authorizing `identity:delete_user(user_id=aead9d71564f4871a5302614aa05bfde)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:29:53.823775 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2ce223-3d4a-4692-a704-ae0d68c1e718 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:29:53.868945 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-fa2ce223-3d4a-4692-a704-ae0d68c1e718 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aead9d71564f4871a5302614aa05bfde'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:29:53.881490 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-fa2ce223-3d4a-4692-a704-ae0d68c1e718 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aead9d71564f4871a5302614aa05bfde'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:29:53.888736 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-fa2ce223-3d4a-4692-a704-ae0d68c1e718 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aead9d71564f4871a5302614aa05bfde'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:29:53.889405 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-fa2ce223-3d4a-4692-a704-ae0d68c1e718 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aead9d71564f4871a5302614aa05bfde'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:29:53.896735 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-fa2ce223-3d4a-4692-a704-ae0d68c1e718 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aead9d71564f4871a5302614aa05bfde'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:29:53.902439 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1165/2355] 10.222.0.15 () {58 vars in 1224 bytes} [Thu Sep 15 12:29:53 2022] DELETE /identity/v3/users/aead9d71564f4871a5302614aa05bfde => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:29:54.172979 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af384ee-4e28-44dc-9025-4559d3e81695 admin admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:29:54.174621 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-7af384ee-4e28-44dc-9025-4559d3e81695 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:29:54.188538 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af384ee-4e28-44dc-9025-4559d3e81695 admin admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:29:54.246533 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbd9317-ddc2-47ab-85a6-2a3b65edda56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:29:54.247254 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbd9317-ddc2-47ab-85a6-2a3b65edda56 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:29:54.247600 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbd9317-ddc2-47ab-85a6-2a3b65edda56 admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:29:54.247939 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbd9317-ddc2-47ab-85a6-2a3b65edda56 admin admin] PATH_INFO: `/v3/projects/fd7fb948652341ebbc789ef959ffa085` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:29:54.249170 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecbd9317-ddc2-47ab-85a6-2a3b65edda56 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fd7fb948652341ebbc789ef959ffa085)` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:29:54.251093 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecbd9317-ddc2-47ab-85a6-2a3b65edda56 admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:29:54.295953 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-ecbd9317-ddc2-47ab-85a6-2a3b65edda56 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fd7fb948652341ebbc789ef959ffa085', 'request_id': 'req-ecbd9317-ddc2-47ab-85a6-2a3b65edda56'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:29:54.304710 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-ecbd9317-ddc2-47ab-85a6-2a3b65edda56 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fd7fb948652341ebbc789ef959ffa085', 'request_id': 'req-ecbd9317-ddc2-47ab-85a6-2a3b65edda56'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:29:54.310528 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-ecbd9317-ddc2-47ab-85a6-2a3b65edda56 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fd7fb948652341ebbc789ef959ffa085', 'request_id': 'req-ecbd9317-ddc2-47ab-85a6-2a3b65edda56'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:29:54.322599 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-ecbd9317-ddc2-47ab-85a6-2a3b65edda56 admin admin] The token cache is being invalidate because project fd7fb948652341ebbc789ef959ffa085 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81449) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 12:29:54.322894 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-ecbd9317-ddc2-47ab-85a6-2a3b65edda56 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:29:54.323550 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-ecbd9317-ddc2-47ab-85a6-2a3b65edda56 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:29:54.325055 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1191/2356] 10.222.0.15 () {58 vars in 1233 bytes} [Thu Sep 15 12:29:54 2022] DELETE /identity/v3/projects/fd7fb948652341ebbc789ef959ffa085 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:29:54.330141 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fda23e8-499c-4e6a-b231-b90f4ee41cf3 None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:29:54.330430 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fda23e8-499c-4e6a-b231-b90f4ee41cf3 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:29:54.330705 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fda23e8-499c-4e6a-b231-b90f4ee41cf3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:29:54.350227 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-5fda23e8-499c-4e6a-b231-b90f4ee41cf3 None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:29:54.351424 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-5fda23e8-499c-4e6a-b231-b90f4ee41cf3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:29:54.408050 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1166/2357] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 12:29:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:29:54.415739 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e5547b-41f8-429c-8085-17902f770326 None None] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:29:54.416029 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e5547b-41f8-429c-8085-17902f770326 None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:29:54.416328 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e5547b-41f8-429c-8085-17902f770326 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:29:54.435852 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.auth.core [None req-67e5547b-41f8-429c-8085-17902f770326 None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81449) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:29:54.437072 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-67e5547b-41f8-429c-8085-17902f770326 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:29:54.445045 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1192/2358] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 12:29:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:29:54.449308 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fda23e8-499c-4e6a-b231-b90f4ee41cf3 None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:29:54.458707 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fda23e8-499c-4e6a-b231-b90f4ee41cf3 None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:29:54.463446 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6781217-0299-4cb3-83e0-cf11d28750e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:29:54.464037 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6781217-0299-4cb3-83e0-cf11d28750e8 admin admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:29:54.464315 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6781217-0299-4cb3-83e0-cf11d28750e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:29:54.464608 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6781217-0299-4cb3-83e0-cf11d28750e8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:29:54.465292 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6781217-0299-4cb3-83e0-cf11d28750e8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:29:54.465596 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6781217-0299-4cb3-83e0-cf11d28750e8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:29:54.467173 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6781217-0299-4cb3-83e0-cf11d28750e8 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:29:54.478904 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1167/2359] 10.222.0.15 () {58 vars in 1153 bytes} [Thu Sep 15 12:29:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:29:54.482979 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e5547b-41f8-429c-8085-17902f770326 None None] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:29:54.492496 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e5547b-41f8-429c-8085-17902f770326 None None] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:29:54.497063 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d6d14e-8a1f-4541-a43b-5febbedcb0c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:29:54.497644 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d6d14e-8a1f-4541-a43b-5febbedcb0c0 admin admin] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:29:54.497916 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d6d14e-8a1f-4541-a43b-5febbedcb0c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:29:54.498185 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d6d14e-8a1f-4541-a43b-5febbedcb0c0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:29:54.498797 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d6d14e-8a1f-4541-a43b-5febbedcb0c0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:29:54.500661 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d6d14e-8a1f-4541-a43b-5febbedcb0c0 admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:29:54.530618 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1193/2360] 10.222.0.15 () {60 vars in 1153 bytes} [Thu Sep 15 12:29:54 2022] POST /identity/v3/projects => generated 377 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 12:29:54.535377 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6781217-0299-4cb3-83e0-cf11d28750e8 admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:29:54.545467 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6781217-0299-4cb3-83e0-cf11d28750e8 admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:29:54.558038 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:29:54.604376 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d78384-03f3-4d48-95a8-f59e6e7e84b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:29:54.605047 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d78384-03f3-4d48-95a8-f59e6e7e84b4 admin admin] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:29:54.605315 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d78384-03f3-4d48-95a8-f59e6e7e84b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:29:54.605591 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d78384-03f3-4d48-95a8-f59e6e7e84b4 admin admin] PATH_INFO: `/v3/users` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:29:54.606253 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d78384-03f3-4d48-95a8-f59e6e7e84b4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:29:54.608234 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d78384-03f3-4d48-95a8-f59e6e7e84b4 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:29:54.630758 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1168/2361] 10.222.0.15 () {60 vars in 1144 bytes} [Thu Sep 15 12:29:54 2022] POST /identity/v3/users => generated 289 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 12:29:54.634776 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d6d14e-8a1f-4541-a43b-5febbedcb0c0 admin admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:29:54.645525 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d6d14e-8a1f-4541-a43b-5febbedcb0c0 admin admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:29:54.650335 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f77443-3f71-4bbc-894a-2787e4e2da0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:29:54.650930 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f77443-3f71-4bbc-894a-2787e4e2da0e admin admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:29:54.651205 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f77443-3f71-4bbc-894a-2787e4e2da0e admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:29:54.651472 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f77443-3f71-4bbc-894a-2787e4e2da0e admin admin] PATH_INFO: `/v3/roles` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:29:54.652105 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f77443-3f71-4bbc-894a-2787e4e2da0e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:29:54.653703 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f77443-3f71-4bbc-894a-2787e4e2da0e admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:29:54.664927 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1194/2362] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 12:29:54 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 12:29:54.668787 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d78384-03f3-4d48-95a8-f59e6e7e84b4 admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:29:54.679523 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d78384-03f3-4d48-95a8-f59e6e7e84b4 admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:29:54.684143 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df144e0-63db-4abb-a5dc-dd60ecc4b2c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:29:54.684726 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df144e0-63db-4abb-a5dc-dd60ecc4b2c9 admin admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:29:54.685049 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df144e0-63db-4abb-a5dc-dd60ecc4b2c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:29:54.685391 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df144e0-63db-4abb-a5dc-dd60ecc4b2c9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:29:54.686114 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df144e0-63db-4abb-a5dc-dd60ecc4b2c9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:29:54.687768 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df144e0-63db-4abb-a5dc-dd60ecc4b2c9 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:29:54.698676 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1169/2363] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 12:29:54 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 12:29:54.702613 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f77443-3f71-4bbc-894a-2787e4e2da0e admin admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:29:54.713024 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f77443-3f71-4bbc-894a-2787e4e2da0e admin admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:29:54.717483 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca6b78d-3e15-42a1-a41b-3907c4d887fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:29:54.718120 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca6b78d-3e15-42a1-a41b-3907c4d887fb admin admin] REQUEST_METHOD: `PUT` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:29:54.718398 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca6b78d-3e15-42a1-a41b-3907c4d887fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:29:54.718664 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca6b78d-3e15-42a1-a41b-3907c4d887fb admin admin] PATH_INFO: `/v3/projects/746ef206242c4ef4b87600a3b3a3f061/users/82e6b87d500c42ef839773f40cd7506d/roles/0679d6628a25438d90dc4bdc0401eda2` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:29:54.732588 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca6b78d-3e15-42a1-a41b-3907c4d887fb admin admin] RBAC: Authorizing `identity:create_grant(project_id=746ef206242c4ef4b87600a3b3a3f061, user_id=82e6b87d500c42ef839773f40cd7506d, role_id=0679d6628a25438d90dc4bdc0401eda2)` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:29:54.735236 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca6b78d-3e15-42a1-a41b-3907c4d887fb admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:29:54.748169 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1195/2364] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 15 12:29:54 2022] PUT /identity/v3/projects/746ef206242c4ef4b87600a3b3a3f061/users/82e6b87d500c42ef839773f40cd7506d/roles/0679d6628a25438d90dc4bdc0401eda2 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 12:29:54.759610 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca3b18c-b515-4e78-bf60-d0a2db6584c3 None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:29:54.759927 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca3b18c-b515-4e78-bf60-d0a2db6584c3 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:29:54.760200 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca3b18c-b515-4e78-bf60-d0a2db6584c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:29:54.787814 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-7ca3b18c-b515-4e78-bf60-d0a2db6584c3 None None] MFA Rules not processed for user `82e6b87d500c42ef839773f40cd7506d`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:29:54.789069 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-7ca3b18c-b515-4e78-bf60-d0a2db6584c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:29:54.851145 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1170/2365] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 12:29:54 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:29:54.855584 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca6b78d-3e15-42a1-a41b-3907c4d887fb admin admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:29:54.875354 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:29:54.919283 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca6b78d-3e15-42a1-a41b-3907c4d887fb admin admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:29:54.924253 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e10608-5cdb-4c72-a8fe-3d5ed9f2f502 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:29:54.924988 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e10608-5cdb-4c72-a8fe-3d5ed9f2f502 admin admin] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:29:54.925268 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e10608-5cdb-4c72-a8fe-3d5ed9f2f502 admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:29:54.925534 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e10608-5cdb-4c72-a8fe-3d5ed9f2f502 admin admin] PATH_INFO: `/v3/projects` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:29:54.926155 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e10608-5cdb-4c72-a8fe-3d5ed9f2f502 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:29:54.928029 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e10608-5cdb-4c72-a8fe-3d5ed9f2f502 admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:29:54.953086 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1196/2366] 10.222.0.15 () {60 vars in 1153 bytes} [Thu Sep 15 12:29:54 2022] POST /identity/v3/projects => generated 377 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 15 12:29:54.957626 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca3b18c-b515-4e78-bf60-d0a2db6584c3 None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:29:54.968635 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca3b18c-b515-4e78-bf60-d0a2db6584c3 None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:29:55.025366 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0b77c2-4b7e-4d19-87e3-aab0742f80e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:29:55.025975 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0b77c2-4b7e-4d19-87e3-aab0742f80e7 admin admin] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:29:55.026251 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0b77c2-4b7e-4d19-87e3-aab0742f80e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:29:55.026523 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0b77c2-4b7e-4d19-87e3-aab0742f80e7 admin admin] PATH_INFO: `/v3/users` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:29:55.027169 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b0b77c2-4b7e-4d19-87e3-aab0742f80e7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:29:55.029114 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b0b77c2-4b7e-4d19-87e3-aab0742f80e7 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:29:55.050656 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1171/2367] 10.222.0.15 () {60 vars in 1144 bytes} [Thu Sep 15 12:29:54 2022] POST /identity/v3/users => generated 289 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 12:29:55.054767 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e10608-5cdb-4c72-a8fe-3d5ed9f2f502 admin admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:29:55.065683 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e10608-5cdb-4c72-a8fe-3d5ed9f2f502 admin admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:29:55.070380 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9225235a-c115-42b9-ab9c-ef60bd8705c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:29:55.070979 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9225235a-c115-42b9-ab9c-ef60bd8705c0 admin admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:29:55.071260 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9225235a-c115-42b9-ab9c-ef60bd8705c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:29:55.071530 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9225235a-c115-42b9-ab9c-ef60bd8705c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:29:55.072167 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9225235a-c115-42b9-ab9c-ef60bd8705c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:29:55.073785 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9225235a-c115-42b9-ab9c-ef60bd8705c0 admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:29:55.084072 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1197/2368] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 12:29:55 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 12:29:55.088058 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0b77c2-4b7e-4d19-87e3-aab0742f80e7 admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:29:55.097889 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0b77c2-4b7e-4d19-87e3-aab0742f80e7 admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:29:55.102499 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1a53c1-bfb7-4f22-9a90-7459b665e209 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:29:55.103091 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1a53c1-bfb7-4f22-9a90-7459b665e209 admin admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:29:55.103371 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1a53c1-bfb7-4f22-9a90-7459b665e209 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:29:55.103641 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1a53c1-bfb7-4f22-9a90-7459b665e209 admin admin] PATH_INFO: `/v3/roles` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:29:55.104274 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1a53c1-bfb7-4f22-9a90-7459b665e209 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:29:55.105889 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1a53c1-bfb7-4f22-9a90-7459b665e209 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:29:55.116019 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1172/2369] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 12:29:55 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 12:29:55.119980 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9225235a-c115-42b9-ab9c-ef60bd8705c0 admin admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:29:55.130490 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9225235a-c115-42b9-ab9c-ef60bd8705c0 admin admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:29:55.135157 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3098de-0b09-444b-9a1f-5493b8f2c064 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:29:55.135802 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3098de-0b09-444b-9a1f-5493b8f2c064 admin admin] REQUEST_METHOD: `PUT` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:29:55.136082 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3098de-0b09-444b-9a1f-5493b8f2c064 admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:29:55.136355 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3098de-0b09-444b-9a1f-5493b8f2c064 admin admin] PATH_INFO: `/v3/projects/ccfc31fc7ab6400baeedb1916dbd6d23/users/16f3c9135b4f4eb18554c1b3a0e14266/roles/0679d6628a25438d90dc4bdc0401eda2` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:29:55.150483 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3098de-0b09-444b-9a1f-5493b8f2c064 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ccfc31fc7ab6400baeedb1916dbd6d23, user_id=16f3c9135b4f4eb18554c1b3a0e14266, role_id=0679d6628a25438d90dc4bdc0401eda2)` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:29:55.153131 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3098de-0b09-444b-9a1f-5493b8f2c064 admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:29:55.165766 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1198/2370] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 15 12:29:55 2022] PUT /identity/v3/projects/ccfc31fc7ab6400baeedb1916dbd6d23/users/16f3c9135b4f4eb18554c1b3a0e14266/roles/0679d6628a25438d90dc4bdc0401eda2 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 12:29:55.176960 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1000b12-0995-4096-8819-3f46a4fc0e96 None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:29:55.176960 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1000b12-0995-4096-8819-3f46a4fc0e96 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:29:55.177146 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1000b12-0995-4096-8819-3f46a4fc0e96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:29:55.185306 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:29:55.208483 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-b1000b12-0995-4096-8819-3f46a4fc0e96 None None] MFA Rules not processed for user `16f3c9135b4f4eb18554c1b3a0e14266`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:29:55.209743 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-b1000b12-0995-4096-8819-3f46a4fc0e96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:29:55.273431 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1173/2371] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 12:29:55 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:29:55.278086 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3098de-0b09-444b-9a1f-5493b8f2c064 admin admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:29:55.338187 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3098de-0b09-444b-9a1f-5493b8f2c064 admin admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:29:55.342746 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b7edf0-d6f8-4b8e-b0f0-1d8a4d9f423d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:29:55.343345 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b7edf0-d6f8-4b8e-b0f0-1d8a4d9f423d admin admin] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:29:55.343623 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b7edf0-d6f8-4b8e-b0f0-1d8a4d9f423d admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:29:55.343893 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b7edf0-d6f8-4b8e-b0f0-1d8a4d9f423d admin admin] PATH_INFO: `/v3/projects` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:29:55.344524 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b7edf0-d6f8-4b8e-b0f0-1d8a4d9f423d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:29:55.346969 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b7edf0-d6f8-4b8e-b0f0-1d8a4d9f423d admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:29:55.373412 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1199/2372] 10.222.0.15 () {60 vars in 1153 bytes} [Thu Sep 15 12:29:55 2022] POST /identity/v3/projects => generated 377 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 12:29:55.377238 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1000b12-0995-4096-8819-3f46a4fc0e96 None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:29:55.388141 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1000b12-0995-4096-8819-3f46a4fc0e96 None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:29:55.448160 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4698bd-1f07-492d-b710-b3c63dc38c65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:29:55.448777 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4698bd-1f07-492d-b710-b3c63dc38c65 admin admin] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:29:55.449075 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4698bd-1f07-492d-b710-b3c63dc38c65 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:29:55.449346 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4698bd-1f07-492d-b710-b3c63dc38c65 admin admin] PATH_INFO: `/v3/users` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:29:55.449972 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4698bd-1f07-492d-b710-b3c63dc38c65 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:29:55.451862 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4698bd-1f07-492d-b710-b3c63dc38c65 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:29:55.473413 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1174/2373] 10.222.0.15 () {60 vars in 1144 bytes} [Thu Sep 15 12:29:55 2022] POST /identity/v3/users => generated 295 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 12:29:55.477431 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b7edf0-d6f8-4b8e-b0f0-1d8a4d9f423d admin admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:29:55.488173 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b7edf0-d6f8-4b8e-b0f0-1d8a4d9f423d admin admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:29:55.493013 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b033c533-abaf-4016-9406-df643c3da81a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:29:55.493611 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b033c533-abaf-4016-9406-df643c3da81a admin admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:29:55.493887 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b033c533-abaf-4016-9406-df643c3da81a admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:29:55.494157 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b033c533-abaf-4016-9406-df643c3da81a admin admin] PATH_INFO: `/v3/roles` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:29:55.494790 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b033c533-abaf-4016-9406-df643c3da81a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:29:55.496370 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b033c533-abaf-4016-9406-df643c3da81a admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:29:55.506747 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1200/2374] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 12:29:55 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 12:29:55.510711 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4698bd-1f07-492d-b710-b3c63dc38c65 admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:29:55.520799 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4698bd-1f07-492d-b710-b3c63dc38c65 admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:29:55.525290 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69ea1bd-866a-47c4-8b74-12553a030864 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:29:55.525928 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69ea1bd-866a-47c4-8b74-12553a030864 admin admin] REQUEST_METHOD: `PUT` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:29:55.526203 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69ea1bd-866a-47c4-8b74-12553a030864 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:29:55.526466 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69ea1bd-866a-47c4-8b74-12553a030864 admin admin] PATH_INFO: `/v3/projects/1707ef2eab50481184f321852926bd6d/users/d32d60e65faf46fd99a6c15ad4366691/roles/9c769b5c82884766b2c510b1f0f1364c` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:29:55.541621 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f69ea1bd-866a-47c4-8b74-12553a030864 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1707ef2eab50481184f321852926bd6d, user_id=d32d60e65faf46fd99a6c15ad4366691, role_id=9c769b5c82884766b2c510b1f0f1364c)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:29:55.544237 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f69ea1bd-866a-47c4-8b74-12553a030864 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:29:55.557182 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1175/2375] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 15 12:29:55 2022] PUT /identity/v3/projects/1707ef2eab50481184f321852926bd6d/users/d32d60e65faf46fd99a6c15ad4366691/roles/9c769b5c82884766b2c510b1f0f1364c => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:29:55.561013 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b033c533-abaf-4016-9406-df643c3da81a admin admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:29:55.570842 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b033c533-abaf-4016-9406-df643c3da81a admin admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:29:55.610023 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:29:55.627960 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92d565a-d520-4225-9595-c8551cc8d193 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:29:55.628564 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92d565a-d520-4225-9595-c8551cc8d193 admin admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:29:55.628874 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92d565a-d520-4225-9595-c8551cc8d193 admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:29:55.629176 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92d565a-d520-4225-9595-c8551cc8d193 admin admin] PATH_INFO: `/v3/roles` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:29:55.629805 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92d565a-d520-4225-9595-c8551cc8d193 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:29:55.631388 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92d565a-d520-4225-9595-c8551cc8d193 admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:29:55.641067 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1201/2376] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 12:29:55 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 12:29:55.645171 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69ea1bd-866a-47c4-8b74-12553a030864 admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:29:55.655165 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69ea1bd-866a-47c4-8b74-12553a030864 admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:29:55.659797 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a865e0-c19b-40d4-a233-aaff32ff4985 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:29:55.660453 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a865e0-c19b-40d4-a233-aaff32ff4985 admin admin] REQUEST_METHOD: `PUT` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:29:55.660791 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a865e0-c19b-40d4-a233-aaff32ff4985 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:29:55.661170 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a865e0-c19b-40d4-a233-aaff32ff4985 admin admin] PATH_INFO: `/v3/projects/1707ef2eab50481184f321852926bd6d/users/d32d60e65faf46fd99a6c15ad4366691/roles/9c769b5c82884766b2c510b1f0f1364c` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:29:55.663064 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a865e0-c19b-40d4-a233-aaff32ff4985 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1707ef2eab50481184f321852926bd6d, user_id=d32d60e65faf46fd99a6c15ad4366691, role_id=9c769b5c82884766b2c510b1f0f1364c)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:29:55.665738 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a865e0-c19b-40d4-a233-aaff32ff4985 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:29:55.677879 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1176/2377] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 15 12:29:55 2022] PUT /identity/v3/projects/1707ef2eab50481184f321852926bd6d/users/d32d60e65faf46fd99a6c15ad4366691/roles/9c769b5c82884766b2c510b1f0f1364c => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:29:55.689863 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449ab07f-c305-4dc3-9610-1c36113410bc None None] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:29:55.690531 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449ab07f-c305-4dc3-9610-1c36113410bc None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:29:55.690831 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449ab07f-c305-4dc3-9610-1c36113410bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:29:55.718069 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.auth.core [None req-449ab07f-c305-4dc3-9610-1c36113410bc None None] MFA Rules not processed for user `d32d60e65faf46fd99a6c15ad4366691`. Rule list: `[]` (Enabled: `True`). {{(pid=81449) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:29:55.719184 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-449ab07f-c305-4dc3-9610-1c36113410bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:29:55.784961 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1202/2378] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 12:29:55 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:29:55.796465 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a865e0-c19b-40d4-a233-aaff32ff4985 admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:29:55.798178 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-f9a865e0-c19b-40d4-a233-aaff32ff4985 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:29:55.850501 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:29:55.856819 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a865e0-c19b-40d4-a233-aaff32ff4985 admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:29:55.860591 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f1adc6-9405-41c8-8c88-d7574e1c5358 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8191d3ccb874529b32f8d0026db629d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '540a32ef88764e8c8d9201ae22f6ddaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:29:55.861302 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f1adc6-9405-41c8-8c88-d7574e1c5358 service cinder] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:29:55.861580 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f1adc6-9405-41c8-8c88-d7574e1c5358 service cinder] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:29:55.861848 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f1adc6-9405-41c8-8c88-d7574e1c5358 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:29:55.918479 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f1adc6-9405-41c8-8c88-d7574e1c5358 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:29:55.919882 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f1adc6-9405-41c8-8c88-d7574e1c5358 service cinder] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:29:55.928378 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1177/2379] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 12:29:55 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 12:29:57.426404 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449ab07f-c305-4dc3-9610-1c36113410bc None None] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:29:57.427855 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-449ab07f-c305-4dc3-9610-1c36113410bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:29:57.439868 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449ab07f-c305-4dc3-9610-1c36113410bc None None] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:29:57.443301 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fe0e85-b80e-4a72-9c4f-d736c8a3f15a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8191d3ccb874529b32f8d0026db629d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '540a32ef88764e8c8d9201ae22f6ddaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:29:57.443890 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fe0e85-b80e-4a72-9c4f-d736c8a3f15a service cinder] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:29:57.444161 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fe0e85-b80e-4a72-9c4f-d736c8a3f15a service cinder] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:29:57.444427 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4fe0e85-b80e-4a72-9c4f-d736c8a3f15a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:29:57.500189 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4fe0e85-b80e-4a72-9c4f-d736c8a3f15a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:29:57.502204 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4fe0e85-b80e-4a72-9c4f-d736c8a3f15a service cinder] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:29:57.511220 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1203/2380] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 12:29:57 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 12:29:58.037716 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f1adc6-9405-41c8-8c88-d7574e1c5358 service cinder] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:29:58.046186 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f1adc6-9405-41c8-8c88-d7574e1c5358 service cinder] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:29:58.050305 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44209ad-2673-42de-8201-6fb66c00e66b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8191d3ccb874529b32f8d0026db629d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '540a32ef88764e8c8d9201ae22f6ddaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:29:58.050905 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44209ad-2673-42de-8201-6fb66c00e66b service cinder] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:29:58.051186 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44209ad-2673-42de-8201-6fb66c00e66b service cinder] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:29:58.051917 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44209ad-2673-42de-8201-6fb66c00e66b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:29:58.060666 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44209ad-2673-42de-8201-6fb66c00e66b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:29:58.062114 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44209ad-2673-42de-8201-6fb66c00e66b service cinder] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:29:58.072118 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1178/2381] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 12:29:58 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 12:30:01.953483 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fe0e85-b80e-4a72-9c4f-d736c8a3f15a service cinder] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:30:02.018237 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4fe0e85-b80e-4a72-9c4f-d736c8a3f15a service cinder] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:30:02.022720 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0135ff6-ab8b-43ab-9619-40701acd75fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:30:02.023349 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0135ff6-ab8b-43ab-9619-40701acd75fe admin admin] REQUEST_METHOD: `DELETE` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:30:02.023643 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0135ff6-ab8b-43ab-9619-40701acd75fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:30:02.023901 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0135ff6-ab8b-43ab-9619-40701acd75fe admin admin] PATH_INFO: `/v3/users/82e6b87d500c42ef839773f40cd7506d` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:30:02.025037 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0135ff6-ab8b-43ab-9619-40701acd75fe admin admin] RBAC: Authorizing `identity:delete_user(user_id=82e6b87d500c42ef839773f40cd7506d)` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:30:02.026941 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0135ff6-ab8b-43ab-9619-40701acd75fe admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:30:02.029230 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:30:02.072947 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-e0135ff6-ab8b-43ab-9619-40701acd75fe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '82e6b87d500c42ef839773f40cd7506d'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:30:02.082196 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-e0135ff6-ab8b-43ab-9619-40701acd75fe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '82e6b87d500c42ef839773f40cd7506d'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:30:02.094577 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-e0135ff6-ab8b-43ab-9619-40701acd75fe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '82e6b87d500c42ef839773f40cd7506d'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:30:02.095768 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-e0135ff6-ab8b-43ab-9619-40701acd75fe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '82e6b87d500c42ef839773f40cd7506d'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:30:02.103309 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-e0135ff6-ab8b-43ab-9619-40701acd75fe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '82e6b87d500c42ef839773f40cd7506d'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:30:02.109923 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1204/2382] 10.222.0.15 () {58 vars in 1224 bytes} [Thu Sep 15 12:30:01 2022] DELETE /identity/v3/users/82e6b87d500c42ef839773f40cd7506d => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:30:02.114991 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168fdc0b-768f-4eef-883c-3d46a0719dd1 None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:30:02.115471 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168fdc0b-768f-4eef-883c-3d46a0719dd1 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:30:02.115566 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168fdc0b-768f-4eef-883c-3d46a0719dd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:30:02.157309 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-168fdc0b-768f-4eef-883c-3d46a0719dd1 None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:30:02.158542 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-168fdc0b-768f-4eef-883c-3d46a0719dd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:30:02.216838 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1179/2383] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 12:30:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:30:02.228054 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0135ff6-ab8b-43ab-9619-40701acd75fe admin admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:30:02.229581 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-e0135ff6-ab8b-43ab-9619-40701acd75fe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:30:02.299790 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0135ff6-ab8b-43ab-9619-40701acd75fe admin admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:30:02.305127 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17698a73-4405-4a55-a398-7f6af00b0182 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfbb4d6809064f9cb9e06626ed7d6814', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '540a32ef88764e8c8d9201ae22f6ddaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:30:02.306150 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17698a73-4405-4a55-a398-7f6af00b0182 service neutron] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:30:02.306435 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17698a73-4405-4a55-a398-7f6af00b0182 service neutron] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:30:02.306711 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17698a73-4405-4a55-a398-7f6af00b0182 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:30:02.317932 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17698a73-4405-4a55-a398-7f6af00b0182 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:30:02.319645 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17698a73-4405-4a55-a398-7f6af00b0182 service neutron] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:30:02.332043 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1205/2384] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 12:30:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 12:30:02.724399 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168fdc0b-768f-4eef-883c-3d46a0719dd1 None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:30:02.726311 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-168fdc0b-768f-4eef-883c-3d46a0719dd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:30:02.739907 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168fdc0b-768f-4eef-883c-3d46a0719dd1 None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:30:02.747253 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95603b11-28a0-4a9d-b336-38595d2354b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:30:02.747909 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95603b11-28a0-4a9d-b336-38595d2354b1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:30:02.748189 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95603b11-28a0-4a9d-b336-38595d2354b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:30:02.748462 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95603b11-28a0-4a9d-b336-38595d2354b1 admin admin] PATH_INFO: `/v3/projects/746ef206242c4ef4b87600a3b3a3f061` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:30:02.749632 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95603b11-28a0-4a9d-b336-38595d2354b1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=746ef206242c4ef4b87600a3b3a3f061)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:30:02.751578 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95603b11-28a0-4a9d-b336-38595d2354b1 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:30:02.801363 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-95603b11-28a0-4a9d-b336-38595d2354b1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '746ef206242c4ef4b87600a3b3a3f061', 'request_id': 'req-95603b11-28a0-4a9d-b336-38595d2354b1'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:30:02.812903 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-95603b11-28a0-4a9d-b336-38595d2354b1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '746ef206242c4ef4b87600a3b3a3f061', 'request_id': 'req-95603b11-28a0-4a9d-b336-38595d2354b1'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:30:02.820103 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-95603b11-28a0-4a9d-b336-38595d2354b1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '746ef206242c4ef4b87600a3b3a3f061', 'request_id': 'req-95603b11-28a0-4a9d-b336-38595d2354b1'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:30:02.833200 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-95603b11-28a0-4a9d-b336-38595d2354b1 admin admin] The token cache is being invalidate because project 746ef206242c4ef4b87600a3b3a3f061 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81448) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 12:30:02.833498 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-95603b11-28a0-4a9d-b336-38595d2354b1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:30:02.834178 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-95603b11-28a0-4a9d-b336-38595d2354b1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:30:02.835882 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1180/2385] 10.222.0.15 () {58 vars in 1233 bytes} [Thu Sep 15 12:30:02 2022] DELETE /identity/v3/projects/746ef206242c4ef4b87600a3b3a3f061 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:30:02.839914 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17698a73-4405-4a55-a398-7f6af00b0182 service neutron] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:30:02.850520 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17698a73-4405-4a55-a398-7f6af00b0182 service neutron] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:30:02.907961 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea31f08b-72dd-49bf-a695-489fc23613b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:30:02.908642 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea31f08b-72dd-49bf-a695-489fc23613b1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:30:02.908962 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea31f08b-72dd-49bf-a695-489fc23613b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:30:02.909239 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea31f08b-72dd-49bf-a695-489fc23613b1 admin admin] PATH_INFO: `/v3/users/16f3c9135b4f4eb18554c1b3a0e14266` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:30:02.910385 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea31f08b-72dd-49bf-a695-489fc23613b1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=16f3c9135b4f4eb18554c1b3a0e14266)` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:30:02.912304 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea31f08b-72dd-49bf-a695-489fc23613b1 admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:30:02.940942 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:30:02.967992 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-ea31f08b-72dd-49bf-a695-489fc23613b1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '16f3c9135b4f4eb18554c1b3a0e14266'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:30:02.982171 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-ea31f08b-72dd-49bf-a695-489fc23613b1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '16f3c9135b4f4eb18554c1b3a0e14266'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:30:02.994580 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-ea31f08b-72dd-49bf-a695-489fc23613b1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '16f3c9135b4f4eb18554c1b3a0e14266'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:30:02.995167 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-ea31f08b-72dd-49bf-a695-489fc23613b1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '16f3c9135b4f4eb18554c1b3a0e14266'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:30:03.002970 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-ea31f08b-72dd-49bf-a695-489fc23613b1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '16f3c9135b4f4eb18554c1b3a0e14266'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:30:03.009593 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1206/2386] 10.222.0.15 () {58 vars in 1224 bytes} [Thu Sep 15 12:30:02 2022] DELETE /identity/v3/users/16f3c9135b4f4eb18554c1b3a0e14266 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:30:03.083357 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95603b11-28a0-4a9d-b336-38595d2354b1 admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:30:03.084572 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-95603b11-28a0-4a9d-b336-38595d2354b1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:30:03.101460 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:30:03.216768 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95603b11-28a0-4a9d-b336-38595d2354b1 admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:30:03.396499 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f234821-e92f-4b2c-b3bb-0b9137e386d8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eface89e261840a2abb4206be26455c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '540a32ef88764e8c8d9201ae22f6ddaf', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:30:03.397183 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f234821-e92f-4b2c-b3bb-0b9137e386d8 service nova] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:30:03.397462 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f234821-e92f-4b2c-b3bb-0b9137e386d8 service nova] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:30:03.398409 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f234821-e92f-4b2c-b3bb-0b9137e386d8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:30:03.401141 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-1f234821-e92f-4b2c-b3bb-0b9137e386d8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:30:03.421635 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f234821-e92f-4b2c-b3bb-0b9137e386d8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:30:03.423803 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f234821-e92f-4b2c-b3bb-0b9137e386d8 service nova] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:30:03.443798 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1181/2387] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 12:30:03 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 15 12:30:03.492364 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea31f08b-72dd-49bf-a695-489fc23613b1 admin admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:30:03.494143 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-ea31f08b-72dd-49bf-a695-489fc23613b1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:30:03.584436 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea31f08b-72dd-49bf-a695-489fc23613b1 admin admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:30:03.589773 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5146e2-b1e4-47d8-8667-6d551e15b9d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:30:03.590546 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5146e2-b1e4-47d8-8667-6d551e15b9d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:30:03.590910 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5146e2-b1e4-47d8-8667-6d551e15b9d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:30:03.591385 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5146e2-b1e4-47d8-8667-6d551e15b9d5 admin admin] PATH_INFO: `/v3/projects/ccfc31fc7ab6400baeedb1916dbd6d23` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:30:03.592576 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5146e2-b1e4-47d8-8667-6d551e15b9d5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ccfc31fc7ab6400baeedb1916dbd6d23)` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:30:03.595743 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5146e2-b1e4-47d8-8667-6d551e15b9d5 admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:30:03.655589 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-7e5146e2-b1e4-47d8-8667-6d551e15b9d5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ccfc31fc7ab6400baeedb1916dbd6d23', 'request_id': 'req-7e5146e2-b1e4-47d8-8667-6d551e15b9d5'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:30:03.666288 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-7e5146e2-b1e4-47d8-8667-6d551e15b9d5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ccfc31fc7ab6400baeedb1916dbd6d23', 'request_id': 'req-7e5146e2-b1e4-47d8-8667-6d551e15b9d5'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:30:03.674281 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-7e5146e2-b1e4-47d8-8667-6d551e15b9d5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ccfc31fc7ab6400baeedb1916dbd6d23', 'request_id': 'req-7e5146e2-b1e4-47d8-8667-6d551e15b9d5'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:30:03.688242 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-7e5146e2-b1e4-47d8-8667-6d551e15b9d5 admin admin] The token cache is being invalidate because project ccfc31fc7ab6400baeedb1916dbd6d23 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81449) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 12:30:03.689198 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-7e5146e2-b1e4-47d8-8667-6d551e15b9d5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:30:03.689868 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-7e5146e2-b1e4-47d8-8667-6d551e15b9d5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:30:03.691377 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1207/2388] 10.222.0.15 () {58 vars in 1233 bytes} [Thu Sep 15 12:30:03 2022] DELETE /identity/v3/projects/ccfc31fc7ab6400baeedb1916dbd6d23 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:30:03.695540 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f234821-e92f-4b2c-b3bb-0b9137e386d8 service nova] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:30:03.706866 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f234821-e92f-4b2c-b3bb-0b9137e386d8 service nova] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:30:03.786928 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271f7012-78fe-4c8e-a46b-4af683674253 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:30:03.787584 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271f7012-78fe-4c8e-a46b-4af683674253 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:30:03.787861 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271f7012-78fe-4c8e-a46b-4af683674253 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:30:03.788380 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271f7012-78fe-4c8e-a46b-4af683674253 admin admin] PATH_INFO: `/v3/users/d32d60e65faf46fd99a6c15ad4366691` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:30:03.793658 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-271f7012-78fe-4c8e-a46b-4af683674253 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d32d60e65faf46fd99a6c15ad4366691)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:30:03.799051 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-271f7012-78fe-4c8e-a46b-4af683674253 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:30:03.875116 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:30:03.902671 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-271f7012-78fe-4c8e-a46b-4af683674253 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd32d60e65faf46fd99a6c15ad4366691'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:30:03.916929 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-271f7012-78fe-4c8e-a46b-4af683674253 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd32d60e65faf46fd99a6c15ad4366691'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:30:03.925586 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-271f7012-78fe-4c8e-a46b-4af683674253 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd32d60e65faf46fd99a6c15ad4366691'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:30:03.926285 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-271f7012-78fe-4c8e-a46b-4af683674253 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd32d60e65faf46fd99a6c15ad4366691'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:30:03.933550 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-271f7012-78fe-4c8e-a46b-4af683674253 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd32d60e65faf46fd99a6c15ad4366691'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:30:03.940556 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1182/2389] 10.222.0.15 () {58 vars in 1224 bytes} [Thu Sep 15 12:30:03 2022] DELETE /identity/v3/users/d32d60e65faf46fd99a6c15ad4366691 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 12:30:04.231787 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5146e2-b1e4-47d8-8667-6d551e15b9d5 admin admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:30:04.233048 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-7e5146e2-b1e4-47d8-8667-6d551e15b9d5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:30:04.250825 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5146e2-b1e4-47d8-8667-6d551e15b9d5 admin admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:30:04.286001 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:DELETE=13,keystone:SELECT=85,keystone:INSERT=2 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:30:04.321511 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7237902f-010c-49b2-8706-9db71a6e1202 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:30:04.322155 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7237902f-010c-49b2-8706-9db71a6e1202 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:30:04.322436 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7237902f-010c-49b2-8706-9db71a6e1202 admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:30:04.322707 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7237902f-010c-49b2-8706-9db71a6e1202 admin admin] PATH_INFO: `/v3/projects/1707ef2eab50481184f321852926bd6d` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:30:04.323847 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7237902f-010c-49b2-8706-9db71a6e1202 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1707ef2eab50481184f321852926bd6d)` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:30:04.325876 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7237902f-010c-49b2-8706-9db71a6e1202 admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:30:04.377005 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-7237902f-010c-49b2-8706-9db71a6e1202 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1707ef2eab50481184f321852926bd6d', 'request_id': 'req-7237902f-010c-49b2-8706-9db71a6e1202'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:30:04.384439 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-7237902f-010c-49b2-8706-9db71a6e1202 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1707ef2eab50481184f321852926bd6d', 'request_id': 'req-7237902f-010c-49b2-8706-9db71a6e1202'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:30:04.390252 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-7237902f-010c-49b2-8706-9db71a6e1202 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1707ef2eab50481184f321852926bd6d', 'request_id': 'req-7237902f-010c-49b2-8706-9db71a6e1202'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:30:04.403805 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-7237902f-010c-49b2-8706-9db71a6e1202 admin admin] The token cache is being invalidate because project 1707ef2eab50481184f321852926bd6d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81449) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 12:30:04.404083 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-7237902f-010c-49b2-8706-9db71a6e1202 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:30:04.404734 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-7237902f-010c-49b2-8706-9db71a6e1202 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:30:04.406441 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1208/2390] 10.222.0.15 () {58 vars in 1233 bytes} [Thu Sep 15 12:30:04 2022] DELETE /identity/v3/projects/1707ef2eab50481184f321852926bd6d => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:30:04.412073 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a193c18f-9475-4909-8dc6-d1295f40b358 None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:30:04.412532 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a193c18f-9475-4909-8dc6-d1295f40b358 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:30:04.412646 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a193c18f-9475-4909-8dc6-d1295f40b358 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:30:04.432805 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-a193c18f-9475-4909-8dc6-d1295f40b358 None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:30:04.434003 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-a193c18f-9475-4909-8dc6-d1295f40b358 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:30:04.495458 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1183/2391] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 12:30:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:30:04.503991 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ddf40c-478a-4f22-bbac-6a5f64a0c7ab None None] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:30:04.504283 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ddf40c-478a-4f22-bbac-6a5f64a0c7ab None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:30:04.504589 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ddf40c-478a-4f22-bbac-6a5f64a0c7ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:30:04.525810 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.auth.core [None req-e9ddf40c-478a-4f22-bbac-6a5f64a0c7ab None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81449) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:30:04.526921 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-e9ddf40c-478a-4f22-bbac-6a5f64a0c7ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:30:04.536512 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1209/2392] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 12:30:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:30:04.540940 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a193c18f-9475-4909-8dc6-d1295f40b358 None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:30:04.551220 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a193c18f-9475-4909-8dc6-d1295f40b358 None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:30:04.555968 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e2f970-4b2a-41fe-9654-16a4899a5a37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:30:04.556581 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e2f970-4b2a-41fe-9654-16a4899a5a37 admin admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:30:04.556899 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e2f970-4b2a-41fe-9654-16a4899a5a37 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:30:04.557172 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e2f970-4b2a-41fe-9654-16a4899a5a37 admin admin] PATH_INFO: `/v3/domains` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:30:04.557969 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e2f970-4b2a-41fe-9654-16a4899a5a37 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:30:04.558274 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e2f970-4b2a-41fe-9654-16a4899a5a37 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:30:04.559864 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e2f970-4b2a-41fe-9654-16a4899a5a37 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:30:04.571764 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1184/2393] 10.222.0.15 () {58 vars in 1153 bytes} [Thu Sep 15 12:30:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:30:04.576047 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ddf40c-478a-4f22-bbac-6a5f64a0c7ab None None] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:30:04.586080 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ddf40c-478a-4f22-bbac-6a5f64a0c7ab None None] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:30:04.591285 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42add71-1594-4813-99f0-38c3c6732f71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:30:04.591887 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42add71-1594-4813-99f0-38c3c6732f71 admin admin] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:30:04.592166 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42add71-1594-4813-99f0-38c3c6732f71 admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:30:04.592435 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42add71-1594-4813-99f0-38c3c6732f71 admin admin] PATH_INFO: `/v3/projects` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:30:04.593107 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42add71-1594-4813-99f0-38c3c6732f71 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:30:04.595526 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42add71-1594-4813-99f0-38c3c6732f71 admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:30:04.622800 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1210/2394] 10.222.0.15 () {60 vars in 1153 bytes} [Thu Sep 15 12:30:04 2022] POST /identity/v3/projects => generated 371 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 12:30:04.626968 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e2f970-4b2a-41fe-9654-16a4899a5a37 admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:30:04.636915 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e2f970-4b2a-41fe-9654-16a4899a5a37 admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:30:04.695647 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f619fc-c396-46dc-aed8-9e99448bb6ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:30:04.696261 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f619fc-c396-46dc-aed8-9e99448bb6ee admin admin] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:30:04.696543 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f619fc-c396-46dc-aed8-9e99448bb6ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:30:04.696846 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f619fc-c396-46dc-aed8-9e99448bb6ee admin admin] PATH_INFO: `/v3/users` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:30:04.697479 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f619fc-c396-46dc-aed8-9e99448bb6ee admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:30:04.699381 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f619fc-c396-46dc-aed8-9e99448bb6ee admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:30:04.729499 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1185/2395] 10.222.0.15 () {60 vars in 1144 bytes} [Thu Sep 15 12:30:04 2022] POST /identity/v3/users => generated 286 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 12:30:04.736613 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42add71-1594-4813-99f0-38c3c6732f71 admin admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:30:04.747207 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42add71-1594-4813-99f0-38c3c6732f71 admin admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:30:04.752150 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f0c183-0de9-4419-a93e-a003266f607f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:30:04.752805 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f0c183-0de9-4419-a93e-a003266f607f admin admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:30:04.753174 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f0c183-0de9-4419-a93e-a003266f607f admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:30:04.753373 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f0c183-0de9-4419-a93e-a003266f607f admin admin] PATH_INFO: `/v3/roles` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:30:04.754007 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f0c183-0de9-4419-a93e-a003266f607f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:30:04.755616 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f0c183-0de9-4419-a93e-a003266f607f admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:30:04.765799 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1211/2396] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 12:30:04 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 12:30:04.770433 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f619fc-c396-46dc-aed8-9e99448bb6ee admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:30:04.780874 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f619fc-c396-46dc-aed8-9e99448bb6ee admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:30:04.785497 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06403809-efb2-43d6-8486-9a9fbc5f0b62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:30:04.786094 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06403809-efb2-43d6-8486-9a9fbc5f0b62 admin admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:30:04.786372 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06403809-efb2-43d6-8486-9a9fbc5f0b62 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:30:04.786637 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06403809-efb2-43d6-8486-9a9fbc5f0b62 admin admin] PATH_INFO: `/v3/roles` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:30:04.787266 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06403809-efb2-43d6-8486-9a9fbc5f0b62 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:30:04.788858 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06403809-efb2-43d6-8486-9a9fbc5f0b62 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:30:04.800500 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1186/2397] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 12:30:04 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 15 12:30:04.804471 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f0c183-0de9-4419-a93e-a003266f607f admin admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:30:04.814423 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f0c183-0de9-4419-a93e-a003266f607f admin admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:30:04.819076 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b3da5f-a55b-4e9f-85d3-8d5588475a46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:30:04.819738 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b3da5f-a55b-4e9f-85d3-8d5588475a46 admin admin] REQUEST_METHOD: `PUT` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:30:04.820017 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b3da5f-a55b-4e9f-85d3-8d5588475a46 admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:30:04.820289 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b3da5f-a55b-4e9f-85d3-8d5588475a46 admin admin] PATH_INFO: `/v3/projects/092e830b85d3479a952d2e4ada193ab3/users/0818d1847d5e42b6a942557f95b09d05/roles/0679d6628a25438d90dc4bdc0401eda2` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:30:04.835955 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b3da5f-a55b-4e9f-85d3-8d5588475a46 admin admin] RBAC: Authorizing `identity:create_grant(project_id=092e830b85d3479a952d2e4ada193ab3, user_id=0818d1847d5e42b6a942557f95b09d05, role_id=0679d6628a25438d90dc4bdc0401eda2)` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:30:04.838911 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b3da5f-a55b-4e9f-85d3-8d5588475a46 admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:30:04.857766 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1212/2398] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 15 12:30:04 2022] PUT /identity/v3/projects/092e830b85d3479a952d2e4ada193ab3/users/0818d1847d5e42b6a942557f95b09d05/roles/0679d6628a25438d90dc4bdc0401eda2 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 12:30:04.863542 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8e8fb5-75f6-4544-b721-6aa6888e684e None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:30:04.863848 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8e8fb5-75f6-4544-b721-6aa6888e684e None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:30:04.864133 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8e8fb5-75f6-4544-b721-6aa6888e684e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:30:04.884713 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-ae8e8fb5-75f6-4544-b721-6aa6888e684e None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:30:04.885946 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-ae8e8fb5-75f6-4544-b721-6aa6888e684e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:30:04.899486 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:30:04.963595 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1187/2399] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 12:30:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:30:04.978997 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b3da5f-a55b-4e9f-85d3-8d5588475a46 admin admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:30:04.980835 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-b2b3da5f-a55b-4e9f-85d3-8d5588475a46 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:30:05.053542 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b3da5f-a55b-4e9f-85d3-8d5588475a46 admin admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:30:05.058393 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b9de13-e909-4e9c-b63d-89856958dd24 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfbb4d6809064f9cb9e06626ed7d6814', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '540a32ef88764e8c8d9201ae22f6ddaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:30:05.059012 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b9de13-e909-4e9c-b63d-89856958dd24 service neutron] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:30:05.060165 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b9de13-e909-4e9c-b63d-89856958dd24 service neutron] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:30:05.060458 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b9de13-e909-4e9c-b63d-89856958dd24 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:30:05.075117 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b9de13-e909-4e9c-b63d-89856958dd24 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:30:05.077233 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b9de13-e909-4e9c-b63d-89856958dd24 service neutron] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:30:05.090977 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1213/2400] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 12:30:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 12:30:06.008992 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44abdc4-80ad-45c4-9f1d-76cd4f713c20 None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:30:06.008992 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44abdc4-80ad-45c4-9f1d-76cd4f713c20 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:30:06.008992 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44abdc4-80ad-45c4-9f1d-76cd4f713c20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:30:06.029049 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-d44abdc4-80ad-45c4-9f1d-76cd4f713c20 None None] MFA Rules not processed for user `0818d1847d5e42b6a942557f95b09d05`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:30:06.030260 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-d44abdc4-80ad-45c4-9f1d-76cd4f713c20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:30:06.101020 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1188/2401] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 12:30:05 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:30:06.112788 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b9de13-e909-4e9c-b63d-89856958dd24 service neutron] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:30:06.114262 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-11b9de13-e909-4e9c-b63d-89856958dd24 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:30:06.119887 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:30:06.176913 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b9de13-e909-4e9c-b63d-89856958dd24 service neutron] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:30:06.180673 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe8643d-ea93-4df7-bc67-86b7d0e81668 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8191d3ccb874529b32f8d0026db629d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '540a32ef88764e8c8d9201ae22f6ddaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:30:06.181309 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe8643d-ea93-4df7-bc67-86b7d0e81668 service cinder] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:30:06.181583 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe8643d-ea93-4df7-bc67-86b7d0e81668 service cinder] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:30:06.181849 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe8643d-ea93-4df7-bc67-86b7d0e81668 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:30:06.191079 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe8643d-ea93-4df7-bc67-86b7d0e81668 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:30:06.192646 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe8643d-ea93-4df7-bc67-86b7d0e81668 service cinder] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:30:06.202410 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1214/2402] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 12:30:06 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 12:30:07.629415 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6078880-2c51-40e9-a19e-9945799a9a99 None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:30:07.629939 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6078880-2c51-40e9-a19e-9945799a9a99 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:30:07.630217 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6078880-2c51-40e9-a19e-9945799a9a99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:30:07.649423 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-b6078880-2c51-40e9-a19e-9945799a9a99 None None] MFA Rules not processed for user `0818d1847d5e42b6a942557f95b09d05`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:30:07.650711 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-b6078880-2c51-40e9-a19e-9945799a9a99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:30:07.658442 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1189/2403] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 12:30:07 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:30:07.928180 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe8643d-ea93-4df7-bc67-86b7d0e81668 service cinder] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:30:07.929653 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-2fe8643d-ea93-4df7-bc67-86b7d0e81668 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:30:07.952634 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe8643d-ea93-4df7-bc67-86b7d0e81668 service cinder] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:30:07.961923 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f92763-a439-4887-ae15-0701ec7e76d3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfbb4d6809064f9cb9e06626ed7d6814', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '540a32ef88764e8c8d9201ae22f6ddaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:30:07.962818 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f92763-a439-4887-ae15-0701ec7e76d3 service neutron] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:30:07.963094 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f92763-a439-4887-ae15-0701ec7e76d3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:30:07.963363 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f92763-a439-4887-ae15-0701ec7e76d3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:30:07.965133 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-57f92763-a439-4887-ae15-0701ec7e76d3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:30:07.976151 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f92763-a439-4887-ae15-0701ec7e76d3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:30:07.978888 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f92763-a439-4887-ae15-0701ec7e76d3 service neutron] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:30:07.996462 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1215/2404] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 12:30:07 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 69 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 12:30:09.700071 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6078880-2c51-40e9-a19e-9945799a9a99 None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:30:09.712880 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6078880-2c51-40e9-a19e-9945799a9a99 None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:30:09.716930 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ec88f6-c16f-4649-9fdb-5d43f9fe5269 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfbb4d6809064f9cb9e06626ed7d6814', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '540a32ef88764e8c8d9201ae22f6ddaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:30:09.717543 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ec88f6-c16f-4649-9fdb-5d43f9fe5269 service neutron] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:30:09.718202 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ec88f6-c16f-4649-9fdb-5d43f9fe5269 service neutron] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:30:09.718652 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ec88f6-c16f-4649-9fdb-5d43f9fe5269 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:30:09.721030 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-56ec88f6-c16f-4649-9fdb-5d43f9fe5269 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:30:09.738839 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ec88f6-c16f-4649-9fdb-5d43f9fe5269 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:30:09.740348 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ec88f6-c16f-4649-9fdb-5d43f9fe5269 service neutron] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:30:09.750487 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1190/2405] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 12:30:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 12:30:13.334571 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f92763-a439-4887-ae15-0701ec7e76d3 service neutron] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:30:13.351202 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f92763-a439-4887-ae15-0701ec7e76d3 service neutron] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:30:13.359423 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81407468-e45a-4225-8ccf-82b59d7b41fe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfbb4d6809064f9cb9e06626ed7d6814', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '540a32ef88764e8c8d9201ae22f6ddaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:30:13.360538 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81407468-e45a-4225-8ccf-82b59d7b41fe service neutron] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:30:13.361828 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81407468-e45a-4225-8ccf-82b59d7b41fe service neutron] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:30:13.362494 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81407468-e45a-4225-8ccf-82b59d7b41fe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:30:13.379207 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-81407468-e45a-4225-8ccf-82b59d7b41fe service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:30:13.403970 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81407468-e45a-4225-8ccf-82b59d7b41fe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:30:13.405954 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81407468-e45a-4225-8ccf-82b59d7b41fe service neutron] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:30:13.426185 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1216/2406] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 12:30:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 12:30:13.726819 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ec88f6-c16f-4649-9fdb-5d43f9fe5269 service neutron] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:30:13.728220 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-56ec88f6-c16f-4649-9fdb-5d43f9fe5269 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:30:13.798768 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ec88f6-c16f-4649-9fdb-5d43f9fe5269 service neutron] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:30:13.803817 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fb17fd-5338-4d4b-b982-2b834e67733e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f64a058bd5440cb5268f228aa6e392', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '540a32ef88764e8c8d9201ae22f6ddaf', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:30:13.804430 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fb17fd-5338-4d4b-b982-2b834e67733e service placement] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:30:13.804713 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fb17fd-5338-4d4b-b982-2b834e67733e service placement] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:30:13.805041 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fb17fd-5338-4d4b-b982-2b834e67733e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:30:13.807507 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-d4fb17fd-5338-4d4b-b982-2b834e67733e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:30:13.823731 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4fb17fd-5338-4d4b-b982-2b834e67733e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:30:13.824300 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81407468-e45a-4225-8ccf-82b59d7b41fe service neutron] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:30:13.825658 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4fb17fd-5338-4d4b-b982-2b834e67733e service placement] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:30:13.826002 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-81407468-e45a-4225-8ccf-82b59d7b41fe service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:30:13.837806 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1191/2407] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 12:30:13 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 12:30:13.841506 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81407468-e45a-4225-8ccf-82b59d7b41fe service neutron] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:30:13.847338 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa03a46-61cb-4aa1-b262-224b33ca9ccd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:30:13.847973 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa03a46-61cb-4aa1-b262-224b33ca9ccd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:30:13.848252 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa03a46-61cb-4aa1-b262-224b33ca9ccd admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:30:13.848523 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa03a46-61cb-4aa1-b262-224b33ca9ccd admin admin] PATH_INFO: `/v3/users/4ead4be9666942f2b81cbb0f9ab69325` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:30:13.850059 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa03a46-61cb-4aa1-b262-224b33ca9ccd admin admin] RBAC: Authorizing `identity:delete_user(user_id=4ead4be9666942f2b81cbb0f9ab69325)` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:30:13.851945 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa03a46-61cb-4aa1-b262-224b33ca9ccd admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:30:13.907621 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-dfa03a46-61cb-4aa1-b262-224b33ca9ccd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4ead4be9666942f2b81cbb0f9ab69325'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:30:13.915726 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-dfa03a46-61cb-4aa1-b262-224b33ca9ccd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4ead4be9666942f2b81cbb0f9ab69325'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:30:13.928198 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-dfa03a46-61cb-4aa1-b262-224b33ca9ccd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4ead4be9666942f2b81cbb0f9ab69325'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:30:13.928889 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-dfa03a46-61cb-4aa1-b262-224b33ca9ccd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4ead4be9666942f2b81cbb0f9ab69325'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:30:13.935918 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-dfa03a46-61cb-4aa1-b262-224b33ca9ccd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4ead4be9666942f2b81cbb0f9ab69325'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:30:13.942431 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1217/2408] 10.222.0.15 () {58 vars in 1224 bytes} [Thu Sep 15 12:30:13 2022] DELETE /identity/v3/users/4ead4be9666942f2b81cbb0f9ab69325 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:30:14.371048 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fb17fd-5338-4d4b-b982-2b834e67733e service placement] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:30:14.381726 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fb17fd-5338-4d4b-b982-2b834e67733e service placement] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:30:14.405908 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:30:14.462832 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0f9e82-2c22-4dc1-9d2c-1124b3fb62b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:30:14.463498 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0f9e82-2c22-4dc1-9d2c-1124b3fb62b1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:30:14.463786 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0f9e82-2c22-4dc1-9d2c-1124b3fb62b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:30:14.464057 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0f9e82-2c22-4dc1-9d2c-1124b3fb62b1 admin admin] PATH_INFO: `/v3/projects/7a670696e63446e4bf597c40c6e72490` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:30:14.465223 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0f9e82-2c22-4dc1-9d2c-1124b3fb62b1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7a670696e63446e4bf597c40c6e72490)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:30:14.467114 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0f9e82-2c22-4dc1-9d2c-1124b3fb62b1 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:30:14.535392 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-8d0f9e82-2c22-4dc1-9d2c-1124b3fb62b1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7a670696e63446e4bf597c40c6e72490', 'request_id': 'req-8d0f9e82-2c22-4dc1-9d2c-1124b3fb62b1'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:30:14.547619 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-8d0f9e82-2c22-4dc1-9d2c-1124b3fb62b1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7a670696e63446e4bf597c40c6e72490', 'request_id': 'req-8d0f9e82-2c22-4dc1-9d2c-1124b3fb62b1'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:30:14.556826 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-8d0f9e82-2c22-4dc1-9d2c-1124b3fb62b1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7a670696e63446e4bf597c40c6e72490', 'request_id': 'req-8d0f9e82-2c22-4dc1-9d2c-1124b3fb62b1'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:30:14.569555 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-8d0f9e82-2c22-4dc1-9d2c-1124b3fb62b1 admin admin] The token cache is being invalidate because project 7a670696e63446e4bf597c40c6e72490 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81448) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 12:30:14.569936 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-8d0f9e82-2c22-4dc1-9d2c-1124b3fb62b1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:30:14.570725 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-8d0f9e82-2c22-4dc1-9d2c-1124b3fb62b1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:30:14.572561 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1192/2409] 10.222.0.15 () {58 vars in 1233 bytes} [Thu Sep 15 12:30:14 2022] DELETE /identity/v3/projects/7a670696e63446e4bf597c40c6e72490 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:30:14.578358 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06c4c2f-599b-40b6-a282-5663b2aaa770 None None] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:30:14.579417 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06c4c2f-599b-40b6-a282-5663b2aaa770 None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:30:14.579644 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06c4c2f-599b-40b6-a282-5663b2aaa770 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:30:14.601049 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.auth.core [None req-e06c4c2f-599b-40b6-a282-5663b2aaa770 None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81449) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:30:14.602454 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-e06c4c2f-599b-40b6-a282-5663b2aaa770 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:30:14.665783 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:30:14.692726 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1218/2410] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 12:30:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:30:14.694801 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b5d44c-07f2-4579-911f-194866dd7a1d None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:30:14.695156 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b5d44c-07f2-4579-911f-194866dd7a1d None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:30:14.695531 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b5d44c-07f2-4579-911f-194866dd7a1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:30:14.717311 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-67b5d44c-07f2-4579-911f-194866dd7a1d None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:30:14.718653 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-67b5d44c-07f2-4579-911f-194866dd7a1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:30:14.727964 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1193/2411] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 12:30:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:30:14.733690 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06c4c2f-599b-40b6-a282-5663b2aaa770 None None] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:30:14.746291 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06c4c2f-599b-40b6-a282-5663b2aaa770 None None] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:30:14.752144 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87107cc3-92e6-4d67-9e0a-c1626c13d6b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:30:14.753015 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87107cc3-92e6-4d67-9e0a-c1626c13d6b0 admin admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:30:14.753395 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87107cc3-92e6-4d67-9e0a-c1626c13d6b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:30:14.753664 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87107cc3-92e6-4d67-9e0a-c1626c13d6b0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:30:14.754309 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87107cc3-92e6-4d67-9e0a-c1626c13d6b0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81449) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:30:14.754611 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87107cc3-92e6-4d67-9e0a-c1626c13d6b0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:30:14.756200 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87107cc3-92e6-4d67-9e0a-c1626c13d6b0 admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:30:14.769972 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1219/2412] 10.222.0.15 () {58 vars in 1153 bytes} [Thu Sep 15 12:30:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:30:14.774504 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b5d44c-07f2-4579-911f-194866dd7a1d None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:30:14.784552 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b5d44c-07f2-4579-911f-194866dd7a1d None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:30:14.789682 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7296cb-76e3-410d-8355-5b83d53776a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:30:14.790479 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7296cb-76e3-410d-8355-5b83d53776a8 admin admin] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:30:14.790853 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7296cb-76e3-410d-8355-5b83d53776a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:30:14.791097 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7296cb-76e3-410d-8355-5b83d53776a8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:30:14.791890 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7296cb-76e3-410d-8355-5b83d53776a8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:30:14.793838 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7296cb-76e3-410d-8355-5b83d53776a8 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:30:14.823836 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1194/2413] 10.222.0.15 () {60 vars in 1153 bytes} [Thu Sep 15 12:30:14 2022] POST /identity/v3/projects => generated 373 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 12:30:14.828104 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87107cc3-92e6-4d67-9e0a-c1626c13d6b0 admin admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:30:14.839010 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87107cc3-92e6-4d67-9e0a-c1626c13d6b0 admin admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:30:14.903593 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715a97e3-6b35-434c-83b0-d94d859cca8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:30:14.904191 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715a97e3-6b35-434c-83b0-d94d859cca8a admin admin] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:30:14.904605 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715a97e3-6b35-434c-83b0-d94d859cca8a admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:30:14.904946 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715a97e3-6b35-434c-83b0-d94d859cca8a admin admin] PATH_INFO: `/v3/users` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:30:14.905648 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-715a97e3-6b35-434c-83b0-d94d859cca8a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:30:14.908442 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-715a97e3-6b35-434c-83b0-d94d859cca8a admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:30:14.932575 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1220/2414] 10.222.0.15 () {60 vars in 1144 bytes} [Thu Sep 15 12:30:14 2022] POST /identity/v3/users => generated 287 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 12:30:14.936996 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7296cb-76e3-410d-8355-5b83d53776a8 admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:30:14.949386 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7296cb-76e3-410d-8355-5b83d53776a8 admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:30:14.955847 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f3398b-ca88-4107-851e-dc511a7b599d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:30:14.956478 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f3398b-ca88-4107-851e-dc511a7b599d admin admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:30:14.957045 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f3398b-ca88-4107-851e-dc511a7b599d admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:30:14.957319 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f3398b-ca88-4107-851e-dc511a7b599d admin admin] PATH_INFO: `/v3/roles` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:30:14.958115 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f3398b-ca88-4107-851e-dc511a7b599d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:30:14.960526 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f3398b-ca88-4107-851e-dc511a7b599d admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:30:14.977925 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715a97e3-6b35-434c-83b0-d94d859cca8a admin admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:30:14.978350 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1195/2415] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 12:30:14 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 12:30:14.988702 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715a97e3-6b35-434c-83b0-d94d859cca8a admin admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:30:14.993684 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b084731-1c54-4cea-926f-879994b69880 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:30:14.994309 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b084731-1c54-4cea-926f-879994b69880 admin admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:30:14.994588 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b084731-1c54-4cea-926f-879994b69880 admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:30:14.994855 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b084731-1c54-4cea-926f-879994b69880 admin admin] PATH_INFO: `/v3/roles` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:30:14.995504 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b084731-1c54-4cea-926f-879994b69880 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:30:14.997183 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b084731-1c54-4cea-926f-879994b69880 admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:30:15.009612 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1221/2416] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 12:30:14 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 12:30:15.014040 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f3398b-ca88-4107-851e-dc511a7b599d admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:30:15.024926 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f3398b-ca88-4107-851e-dc511a7b599d admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:30:15.029641 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b26f504-9c49-4b12-b1fa-30d757ca2ba3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:30:15.030322 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b26f504-9c49-4b12-b1fa-30d757ca2ba3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:30:15.030599 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b26f504-9c49-4b12-b1fa-30d757ca2ba3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:30:15.031073 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b26f504-9c49-4b12-b1fa-30d757ca2ba3 admin admin] PATH_INFO: `/v3/projects/dfa85af8efe5425d88c4cc99910d83f2/users/80dc3908a9044cccb7b07a3919d35d1e/roles/0679d6628a25438d90dc4bdc0401eda2` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:30:15.048377 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b26f504-9c49-4b12-b1fa-30d757ca2ba3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dfa85af8efe5425d88c4cc99910d83f2, user_id=80dc3908a9044cccb7b07a3919d35d1e, role_id=0679d6628a25438d90dc4bdc0401eda2)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:30:15.051070 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b26f504-9c49-4b12-b1fa-30d757ca2ba3 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:30:15.065753 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1196/2417] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 15 12:30:15 2022] PUT /identity/v3/projects/dfa85af8efe5425d88c4cc99910d83f2/users/80dc3908a9044cccb7b07a3919d35d1e/roles/0679d6628a25438d90dc4bdc0401eda2 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:30:15.076284 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afc728b-7d2a-42e4-b64c-0e247a6fe0ec None None] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:30:15.076571 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afc728b-7d2a-42e4-b64c-0e247a6fe0ec None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:30:15.076939 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6afc728b-7d2a-42e4-b64c-0e247a6fe0ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:30:15.106372 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.auth.core [None req-6afc728b-7d2a-42e4-b64c-0e247a6fe0ec None None] MFA Rules not processed for user `80dc3908a9044cccb7b07a3919d35d1e`. Rule list: `[]` (Enabled: `True`). {{(pid=81449) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:30:15.107502 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-6afc728b-7d2a-42e4-b64c-0e247a6fe0ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:30:15.173331 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1222/2418] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 12:30:15 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:30:15.184329 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b26f504-9c49-4b12-b1fa-30d757ca2ba3 admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:30:15.186135 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-1b26f504-9c49-4b12-b1fa-30d757ca2ba3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:30:15.236472 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:30:15.263560 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b26f504-9c49-4b12-b1fa-30d757ca2ba3 admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:30:15.269547 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9decab55-185c-4587-8c9c-48a2cbacd4e1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eface89e261840a2abb4206be26455c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '540a32ef88764e8c8d9201ae22f6ddaf', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:30:15.270161 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9decab55-185c-4587-8c9c-48a2cbacd4e1 service nova] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:30:15.270443 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9decab55-185c-4587-8c9c-48a2cbacd4e1 service nova] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:30:15.270716 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9decab55-185c-4587-8c9c-48a2cbacd4e1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:30:15.278555 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9decab55-185c-4587-8c9c-48a2cbacd4e1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:30:15.280135 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9decab55-185c-4587-8c9c-48a2cbacd4e1 service nova] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:30:15.290307 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1197/2419] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 12:30:15 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 12:30:16.167549 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6afc728b-7d2a-42e4-b64c-0e247a6fe0ec None None] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:30:16.169635 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-6afc728b-7d2a-42e4-b64c-0e247a6fe0ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:30:16.186865 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6afc728b-7d2a-42e4-b64c-0e247a6fe0ec None None] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:30:16.193965 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061580b2-e4f8-4b04-ac50-9665a52a3ecf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eface89e261840a2abb4206be26455c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '540a32ef88764e8c8d9201ae22f6ddaf', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:30:16.194681 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061580b2-e4f8-4b04-ac50-9665a52a3ecf service nova] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:30:16.195009 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061580b2-e4f8-4b04-ac50-9665a52a3ecf service nova] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:30:16.195359 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061580b2-e4f8-4b04-ac50-9665a52a3ecf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:30:16.198439 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-061580b2-e4f8-4b04-ac50-9665a52a3ecf service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:30:16.218563 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061580b2-e4f8-4b04-ac50-9665a52a3ecf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:30:16.220158 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-061580b2-e4f8-4b04-ac50-9665a52a3ecf service nova] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:30:16.234502 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1223/2420] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 12:30:16 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 67 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 12:30:17.386241 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9decab55-185c-4587-8c9c-48a2cbacd4e1 service nova] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:30:17.388231 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-9decab55-185c-4587-8c9c-48a2cbacd4e1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:30:17.458136 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061580b2-e4f8-4b04-ac50-9665a52a3ecf service nova] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:30:17.459994 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-061580b2-e4f8-4b04-ac50-9665a52a3ecf service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:30:17.479375 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9decab55-185c-4587-8c9c-48a2cbacd4e1 service nova] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:30:17.485549 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d7b345-8ce9-4691-9a23-f60ae9bf4c58 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f64a058bd5440cb5268f228aa6e392', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '540a32ef88764e8c8d9201ae22f6ddaf', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:30:17.486319 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d7b345-8ce9-4691-9a23-f60ae9bf4c58 service placement] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:30:17.486595 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d7b345-8ce9-4691-9a23-f60ae9bf4c58 service placement] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:30:17.486855 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d7b345-8ce9-4691-9a23-f60ae9bf4c58 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:30:17.488557 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-b0d7b345-8ce9-4691-9a23-f60ae9bf4c58 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:30:17.501079 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d7b345-8ce9-4691-9a23-f60ae9bf4c58 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:30:17.502666 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d7b345-8ce9-4691-9a23-f60ae9bf4c58 service placement] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:30:17.504141 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:30:17.513829 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1198/2421] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 12:30:17 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 12:30:17.545269 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061580b2-e4f8-4b04-ac50-9665a52a3ecf service nova] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:30:17.551545 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d07a12-3c1e-4c40-86d4-c5d5f216eb58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:30:17.552195 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d07a12-3c1e-4c40-86d4-c5d5f216eb58 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:30:17.552479 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d07a12-3c1e-4c40-86d4-c5d5f216eb58 admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:30:17.552793 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d07a12-3c1e-4c40-86d4-c5d5f216eb58 admin admin] PATH_INFO: `/v3/users/89cc4de76c6a4305af9a9506840e13f6` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:30:17.553928 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d07a12-3c1e-4c40-86d4-c5d5f216eb58 admin admin] RBAC: Authorizing `identity:delete_user(user_id=89cc4de76c6a4305af9a9506840e13f6)` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:30:17.555891 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d07a12-3c1e-4c40-86d4-c5d5f216eb58 admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:30:17.608921 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-e8d07a12-3c1e-4c40-86d4-c5d5f216eb58 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '89cc4de76c6a4305af9a9506840e13f6'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:30:17.618070 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-e8d07a12-3c1e-4c40-86d4-c5d5f216eb58 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '89cc4de76c6a4305af9a9506840e13f6'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:30:17.630892 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-e8d07a12-3c1e-4c40-86d4-c5d5f216eb58 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '89cc4de76c6a4305af9a9506840e13f6'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:30:17.631496 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-e8d07a12-3c1e-4c40-86d4-c5d5f216eb58 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '89cc4de76c6a4305af9a9506840e13f6'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:30:17.638367 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-e8d07a12-3c1e-4c40-86d4-c5d5f216eb58 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '89cc4de76c6a4305af9a9506840e13f6'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:30:17.644247 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1224/2422] 10.222.0.15 () {58 vars in 1224 bytes} [Thu Sep 15 12:30:17 2022] DELETE /identity/v3/users/89cc4de76c6a4305af9a9506840e13f6 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:30:17.822611 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d7b345-8ce9-4691-9a23-f60ae9bf4c58 service placement] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:30:17.833219 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d7b345-8ce9-4691-9a23-f60ae9bf4c58 service placement] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:30:17.892899 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6a51c4-2199-4fe0-b6dd-b59e604ddd07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:30:17.893458 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6a51c4-2199-4fe0-b6dd-b59e604ddd07 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:30:17.893732 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6a51c4-2199-4fe0-b6dd-b59e604ddd07 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:30:17.894001 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6a51c4-2199-4fe0-b6dd-b59e604ddd07 admin admin] PATH_INFO: `/v3/projects/d2780dab9a5d40739fc9adc9a6ce35a1` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:30:17.895100 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f6a51c4-2199-4fe0-b6dd-b59e604ddd07 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d2780dab9a5d40739fc9adc9a6ce35a1)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:30:17.897003 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f6a51c4-2199-4fe0-b6dd-b59e604ddd07 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:30:17.951787 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-3f6a51c4-2199-4fe0-b6dd-b59e604ddd07 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd2780dab9a5d40739fc9adc9a6ce35a1', 'request_id': 'req-3f6a51c4-2199-4fe0-b6dd-b59e604ddd07'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:30:17.964604 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-3f6a51c4-2199-4fe0-b6dd-b59e604ddd07 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd2780dab9a5d40739fc9adc9a6ce35a1', 'request_id': 'req-3f6a51c4-2199-4fe0-b6dd-b59e604ddd07'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:30:17.970814 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:30:17.972435 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-3f6a51c4-2199-4fe0-b6dd-b59e604ddd07 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd2780dab9a5d40739fc9adc9a6ce35a1', 'request_id': 'req-3f6a51c4-2199-4fe0-b6dd-b59e604ddd07'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:30:17.988247 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-3f6a51c4-2199-4fe0-b6dd-b59e604ddd07 admin admin] The token cache is being invalidate because project d2780dab9a5d40739fc9adc9a6ce35a1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81448) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 12:30:17.988536 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-3f6a51c4-2199-4fe0-b6dd-b59e604ddd07 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:30:17.989227 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-3f6a51c4-2199-4fe0-b6dd-b59e604ddd07 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:30:17.990711 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1199/2423] 10.222.0.15 () {58 vars in 1233 bytes} [Thu Sep 15 12:30:17 2022] DELETE /identity/v3/projects/d2780dab9a5d40739fc9adc9a6ce35a1 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:30:17.995646 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654baada-b6e5-4d12-9b18-ca414a9f67b2 None None] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:30:17.996071 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654baada-b6e5-4d12-9b18-ca414a9f67b2 None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:30:17.996619 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654baada-b6e5-4d12-9b18-ca414a9f67b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:30:18.017238 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.auth.core [None req-654baada-b6e5-4d12-9b18-ca414a9f67b2 None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81449) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:30:18.018485 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-654baada-b6e5-4d12-9b18-ca414a9f67b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:30:18.079674 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6a51c4-2199-4fe0-b6dd-b59e604ddd07 admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:30:18.080364 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-3f6a51c4-2199-4fe0-b6dd-b59e604ddd07 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:30:18.084642 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1225/2424] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 12:30:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:30:18.092385 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ede176-b70b-4ce2-8e6b-13783eda91af None None] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:30:18.092673 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ede176-b70b-4ce2-8e6b-13783eda91af None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:30:18.093077 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ede176-b70b-4ce2-8e6b-13783eda91af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:30:18.093312 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6a51c4-2199-4fe0-b6dd-b59e604ddd07 admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:30:18.098545 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064532d9-803d-4816-80c0-9f0eb78b6e3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:30:18.099229 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064532d9-803d-4816-80c0-9f0eb78b6e3d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:30:18.099540 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064532d9-803d-4816-80c0-9f0eb78b6e3d admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:30:18.099813 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064532d9-803d-4816-80c0-9f0eb78b6e3d admin admin] PATH_INFO: `/v3/users/80dc3908a9044cccb7b07a3919d35d1e` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:30:18.100981 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-064532d9-803d-4816-80c0-9f0eb78b6e3d admin admin] RBAC: Authorizing `identity:delete_user(user_id=80dc3908a9044cccb7b07a3919d35d1e)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:30:18.102935 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-064532d9-803d-4816-80c0-9f0eb78b6e3d admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:30:18.112856 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.auth.core [None req-c1ede176-b70b-4ce2-8e6b-13783eda91af None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81449) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:30:18.114057 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-c1ede176-b70b-4ce2-8e6b-13783eda91af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:30:18.121987 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1226/2425] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 12:30:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:30:18.126188 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ede176-b70b-4ce2-8e6b-13783eda91af None None] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:30:18.135706 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ede176-b70b-4ce2-8e6b-13783eda91af None None] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:30:18.140510 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea19159-7486-497f-bdee-ad1a39bac878 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:30:18.141861 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea19159-7486-497f-bdee-ad1a39bac878 admin admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:30:18.142146 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea19159-7486-497f-bdee-ad1a39bac878 admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:30:18.142416 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea19159-7486-497f-bdee-ad1a39bac878 admin admin] PATH_INFO: `/v3/domains` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:30:18.143108 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea19159-7486-497f-bdee-ad1a39bac878 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81449) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:30:18.143355 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea19159-7486-497f-bdee-ad1a39bac878 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:30:18.144978 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea19159-7486-497f-bdee-ad1a39bac878 admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:30:18.151427 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-064532d9-803d-4816-80c0-9f0eb78b6e3d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '80dc3908a9044cccb7b07a3919d35d1e'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:30:18.157208 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1227/2426] 10.222.0.15 () {58 vars in 1153 bytes} [Thu Sep 15 12:30:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 12:30:18.161420 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea19159-7486-497f-bdee-ad1a39bac878 admin admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:30:18.164881 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-064532d9-803d-4816-80c0-9f0eb78b6e3d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '80dc3908a9044cccb7b07a3919d35d1e'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:30:18.171023 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea19159-7486-497f-bdee-ad1a39bac878 admin admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:30:18.173911 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-064532d9-803d-4816-80c0-9f0eb78b6e3d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '80dc3908a9044cccb7b07a3919d35d1e'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:30:18.174539 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-064532d9-803d-4816-80c0-9f0eb78b6e3d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '80dc3908a9044cccb7b07a3919d35d1e'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:30:18.176399 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ebc16a-83c9-4fc2-992b-ede3cb76f3d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:30:18.177057 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ebc16a-83c9-4fc2-992b-ede3cb76f3d2 admin admin] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:30:18.177334 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ebc16a-83c9-4fc2-992b-ede3cb76f3d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:30:18.177607 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ebc16a-83c9-4fc2-992b-ede3cb76f3d2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:30:18.178243 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ebc16a-83c9-4fc2-992b-ede3cb76f3d2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:30:18.180097 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ebc16a-83c9-4fc2-992b-ede3cb76f3d2 admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:30:18.181524 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-064532d9-803d-4816-80c0-9f0eb78b6e3d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '80dc3908a9044cccb7b07a3919d35d1e'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:30:18.187597 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1200/2427] 10.222.0.15 () {58 vars in 1224 bytes} [Thu Sep 15 12:30:18 2022] DELETE /identity/v3/users/80dc3908a9044cccb7b07a3919d35d1e => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:30:18.191610 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a0eafb-1b40-4f65-9206-461593caed12 None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:30:18.191895 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a0eafb-1b40-4f65-9206-461593caed12 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:30:18.192190 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a0eafb-1b40-4f65-9206-461593caed12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:30:18.209403 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1228/2428] 10.222.0.15 () {60 vars in 1153 bytes} [Thu Sep 15 12:30:18 2022] POST /identity/v3/projects => generated 375 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 15 12:30:18.211245 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-92a0eafb-1b40-4f65-9206-461593caed12 None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:30:18.212710 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-92a0eafb-1b40-4f65-9206-461593caed12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:30:18.213772 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ebc16a-83c9-4fc2-992b-ede3cb76f3d2 admin admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:30:18.215324 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-11ebc16a-83c9-4fc2-992b-ede3cb76f3d2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:30:18.228728 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:30:18.278401 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1201/2429] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 12:30:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 12:30:18.290196 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a0eafb-1b40-4f65-9206-461593caed12 None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:30:18.292089 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-92a0eafb-1b40-4f65-9206-461593caed12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:30:18.300922 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ebc16a-83c9-4fc2-992b-ede3cb76f3d2 admin admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:30:18.310509 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7451b1f-4830-445c-87d6-93d85e832e77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:30:18.311132 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7451b1f-4830-445c-87d6-93d85e832e77 admin admin] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:30:18.311412 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7451b1f-4830-445c-87d6-93d85e832e77 admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:30:18.312114 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7451b1f-4830-445c-87d6-93d85e832e77 admin admin] PATH_INFO: `/v3/users` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:30:18.313457 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7451b1f-4830-445c-87d6-93d85e832e77 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:30:18.316469 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7451b1f-4830-445c-87d6-93d85e832e77 admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:30:18.340546 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1229/2430] 10.222.0.15 () {60 vars in 1144 bytes} [Thu Sep 15 12:30:18 2022] POST /identity/v3/users => generated 288 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 15 12:30:18.344368 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7451b1f-4830-445c-87d6-93d85e832e77 admin admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:30:18.355467 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7451b1f-4830-445c-87d6-93d85e832e77 admin admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:30:18.360884 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55718f99-393b-4337-b169-bfb0d04a543e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:30:18.361557 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55718f99-393b-4337-b169-bfb0d04a543e admin admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:30:18.361672 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a0eafb-1b40-4f65-9206-461593caed12 None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:30:18.361859 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55718f99-393b-4337-b169-bfb0d04a543e admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:30:18.362109 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55718f99-393b-4337-b169-bfb0d04a543e admin admin] PATH_INFO: `/v3/roles` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:30:18.362748 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55718f99-393b-4337-b169-bfb0d04a543e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:30:18.364349 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55718f99-393b-4337-b169-bfb0d04a543e admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:30:18.365422 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfc1307-6394-43bc-8dd1-802e57d822e2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfbb4d6809064f9cb9e06626ed7d6814', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '540a32ef88764e8c8d9201ae22f6ddaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:30:18.366568 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfc1307-6394-43bc-8dd1-802e57d822e2 service neutron] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:30:18.366957 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfc1307-6394-43bc-8dd1-802e57d822e2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:30:18.367235 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfc1307-6394-43bc-8dd1-802e57d822e2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:30:18.376835 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1230/2431] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 12:30:18 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 15 12:30:18.377735 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bfc1307-6394-43bc-8dd1-802e57d822e2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:30:18.379172 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bfc1307-6394-43bc-8dd1-802e57d822e2 service neutron] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:30:18.379774 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55718f99-393b-4337-b169-bfb0d04a543e admin admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:30:18.390241 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1202/2432] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 12:30:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 15 12:30:18.390769 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55718f99-393b-4337-b169-bfb0d04a543e admin admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:30:18.395548 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2fe33d2-ef6d-4354-82b2-dab6f16c0806 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:30:18.396088 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fe33d2-ef6d-4354-82b2-dab6f16c0806 admin admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:30:18.396365 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fe33d2-ef6d-4354-82b2-dab6f16c0806 admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:30:18.396633 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fe33d2-ef6d-4354-82b2-dab6f16c0806 admin admin] PATH_INFO: `/v3/roles` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:30:18.397301 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2fe33d2-ef6d-4354-82b2-dab6f16c0806 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:30:18.398899 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2fe33d2-ef6d-4354-82b2-dab6f16c0806 admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:30:18.410154 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1231/2433] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 12:30:18 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 15 12:30:18.414163 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfc1307-6394-43bc-8dd1-802e57d822e2 service neutron] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:30:18.426315 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfc1307-6394-43bc-8dd1-802e57d822e2 service neutron] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:30:18.432888 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c215e2-335a-4246-b738-812a0c36ee76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:30:18.434083 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c215e2-335a-4246-b738-812a0c36ee76 admin admin] REQUEST_METHOD: `PUT` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:30:18.434227 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c215e2-335a-4246-b738-812a0c36ee76 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:30:18.434594 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c215e2-335a-4246-b738-812a0c36ee76 admin admin] PATH_INFO: `/v3/projects/aa1ddf08a3284f55a104ad7ffc100d5f/users/0020220f06d8474dad0f4d7d1d9fa397/roles/0679d6628a25438d90dc4bdc0401eda2` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:30:18.438426 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:30:18.456781 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c215e2-335a-4246-b738-812a0c36ee76 admin admin] RBAC: Authorizing `identity:create_grant(project_id=aa1ddf08a3284f55a104ad7ffc100d5f, user_id=0020220f06d8474dad0f4d7d1d9fa397, role_id=0679d6628a25438d90dc4bdc0401eda2)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:30:18.460030 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c215e2-335a-4246-b738-812a0c36ee76 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:30:18.474910 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1203/2434] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 15 12:30:18 2022] PUT /identity/v3/projects/aa1ddf08a3284f55a104ad7ffc100d5f/users/0020220f06d8474dad0f4d7d1d9fa397/roles/0679d6628a25438d90dc4bdc0401eda2 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 12:30:18.487131 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fdebba-2acc-4c0c-98fa-981c39daa0da None None] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:30:18.488109 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fdebba-2acc-4c0c-98fa-981c39daa0da None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:30:18.488639 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fdebba-2acc-4c0c-98fa-981c39daa0da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:30:18.525808 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.auth.core [None req-81fdebba-2acc-4c0c-98fa-981c39daa0da None None] MFA Rules not processed for user `0020220f06d8474dad0f4d7d1d9fa397`. Rule list: `[]` (Enabled: `True`). {{(pid=81449) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:30:18.527184 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-81fdebba-2acc-4c0c-98fa-981c39daa0da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:30:18.609513 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1232/2435] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 12:30:18 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:30:18.613988 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c215e2-335a-4246-b738-812a0c36ee76 admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:30:18.722299 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c215e2-335a-4246-b738-812a0c36ee76 admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:30:18.729135 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6bd25ee-f748-4d1f-ab2b-2fb85b6d6333 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:30:18.731524 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bd25ee-f748-4d1f-ab2b-2fb85b6d6333 admin admin] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:30:18.732876 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bd25ee-f748-4d1f-ab2b-2fb85b6d6333 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:30:18.733236 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bd25ee-f748-4d1f-ab2b-2fb85b6d6333 admin admin] PATH_INFO: `/v3/projects` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:30:18.733965 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6bd25ee-f748-4d1f-ab2b-2fb85b6d6333 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:30:18.736159 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6bd25ee-f748-4d1f-ab2b-2fb85b6d6333 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:30:18.782386 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1204/2436] 10.222.0.15 () {60 vars in 1153 bytes} [Thu Sep 15 12:30:18 2022] POST /identity/v3/projects => generated 373 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 12:30:18.786904 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fdebba-2acc-4c0c-98fa-981c39daa0da None None] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:30:18.798450 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fdebba-2acc-4c0c-98fa-981c39daa0da None None] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:30:18.879270 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6bd25ee-f748-4d1f-ab2b-2fb85b6d6333 admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:30:18.882561 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-e6bd25ee-f748-4d1f-ab2b-2fb85b6d6333 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:30:18.895226 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b578e568-53f4-4fd1-adb9-f5f47c4c7384 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:30:18.896885 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b578e568-53f4-4fd1-adb9-f5f47c4c7384 admin admin] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:30:18.897232 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b578e568-53f4-4fd1-adb9-f5f47c4c7384 admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:30:18.897448 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b578e568-53f4-4fd1-adb9-f5f47c4c7384 admin admin] PATH_INFO: `/v3/users` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:30:18.898426 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b578e568-53f4-4fd1-adb9-f5f47c4c7384 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:30:18.907362 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b578e568-53f4-4fd1-adb9-f5f47c4c7384 admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:30:18.953851 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:30:18.963256 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1233/2437] 10.222.0.15 () {60 vars in 1144 bytes} [Thu Sep 15 12:30:18 2022] POST /identity/v3/users => generated 293 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 15 12:30:18.964528 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b578e568-53f4-4fd1-adb9-f5f47c4c7384 admin admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:30:18.981628 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b578e568-53f4-4fd1-adb9-f5f47c4c7384 admin admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:30:18.987641 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d0145a-89d5-4882-b08a-27955124d3f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:30:18.988270 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d0145a-89d5-4882-b08a-27955124d3f9 admin admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:30:18.988550 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d0145a-89d5-4882-b08a-27955124d3f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:30:18.989389 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d0145a-89d5-4882-b08a-27955124d3f9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:30:18.989524 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d0145a-89d5-4882-b08a-27955124d3f9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:30:18.991303 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d0145a-89d5-4882-b08a-27955124d3f9 admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:30:19.003100 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1234/2438] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 12:30:18 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 12:30:19.003770 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6bd25ee-f748-4d1f-ab2b-2fb85b6d6333 admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:30:19.007630 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d0145a-89d5-4882-b08a-27955124d3f9 admin admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:30:19.009233 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef161cd-2d48-4c7c-b370-3cc1b0018fcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:30:19.010234 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef161cd-2d48-4c7c-b370-3cc1b0018fcd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:30:19.010584 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef161cd-2d48-4c7c-b370-3cc1b0018fcd admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:30:19.011088 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef161cd-2d48-4c7c-b370-3cc1b0018fcd admin admin] PATH_INFO: `/v3/projects/dfa85af8efe5425d88c4cc99910d83f2` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:30:19.012298 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef161cd-2d48-4c7c-b370-3cc1b0018fcd admin admin] RBAC: Authorizing `identity:delete_project(project_id=dfa85af8efe5425d88c4cc99910d83f2)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:30:19.014350 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef161cd-2d48-4c7c-b370-3cc1b0018fcd admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:30:19.023188 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d0145a-89d5-4882-b08a-27955124d3f9 admin admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:30:19.028950 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b1042f-ba16-4527-a9ff-57c423beafb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:30:19.029947 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b1042f-ba16-4527-a9ff-57c423beafb6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:30:19.030458 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b1042f-ba16-4527-a9ff-57c423beafb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:30:19.030969 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b1042f-ba16-4527-a9ff-57c423beafb6 admin admin] PATH_INFO: `/v3/projects/9a27904949a047e08aea23e5b60dcbc3/users/32d0c2cbec444dcfad46dece109c27bf/roles/9c769b5c82884766b2c510b1f0f1364c` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:30:19.050773 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b1042f-ba16-4527-a9ff-57c423beafb6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9a27904949a047e08aea23e5b60dcbc3, user_id=32d0c2cbec444dcfad46dece109c27bf, role_id=9c769b5c82884766b2c510b1f0f1364c)` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:30:19.054970 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b1042f-ba16-4527-a9ff-57c423beafb6 admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:30:19.085541 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1235/2439] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 15 12:30:19 2022] PUT /identity/v3/projects/9a27904949a047e08aea23e5b60dcbc3/users/32d0c2cbec444dcfad46dece109c27bf/roles/9c769b5c82884766b2c510b1f0f1364c => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:30:19.089981 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-9ef161cd-2d48-4c7c-b370-3cc1b0018fcd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dfa85af8efe5425d88c4cc99910d83f2', 'request_id': 'req-9ef161cd-2d48-4c7c-b370-3cc1b0018fcd'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:30:19.090864 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b1042f-ba16-4527-a9ff-57c423beafb6 admin admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:30:19.100990 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-9ef161cd-2d48-4c7c-b370-3cc1b0018fcd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dfa85af8efe5425d88c4cc99910d83f2', 'request_id': 'req-9ef161cd-2d48-4c7c-b370-3cc1b0018fcd'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:30:19.126670 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-9ef161cd-2d48-4c7c-b370-3cc1b0018fcd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dfa85af8efe5425d88c4cc99910d83f2', 'request_id': 'req-9ef161cd-2d48-4c7c-b370-3cc1b0018fcd'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:30:19.135731 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:30:19.141686 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-9ef161cd-2d48-4c7c-b370-3cc1b0018fcd admin admin] The token cache is being invalidate because project dfa85af8efe5425d88c4cc99910d83f2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81448) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 12:30:19.142535 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-9ef161cd-2d48-4c7c-b370-3cc1b0018fcd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:30:19.144228 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-9ef161cd-2d48-4c7c-b370-3cc1b0018fcd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:30:19.146359 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1205/2440] 10.222.0.15 () {58 vars in 1233 bytes} [Thu Sep 15 12:30:18 2022] DELETE /identity/v3/projects/dfa85af8efe5425d88c4cc99910d83f2 => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 12:30:19.151790 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b4fb5f-9095-472d-8b58-237af6ec86ae None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:30:19.152289 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b4fb5f-9095-472d-8b58-237af6ec86ae None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:30:19.152727 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b4fb5f-9095-472d-8b58-237af6ec86ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:30:19.173692 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-59b4fb5f-9095-472d-8b58-237af6ec86ae None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:30:19.174848 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-59b4fb5f-9095-472d-8b58-237af6ec86ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:30:19.179226 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b1042f-ba16-4527-a9ff-57c423beafb6 admin admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:30:19.185673 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc45fc0-c488-4b29-bdf0-9b8bac57ae15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:30:19.185673 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc45fc0-c488-4b29-bdf0-9b8bac57ae15 admin admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:30:19.199841 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1206/2441] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 12:30:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 12:30:19.201801 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e16474-a951-474c-8b1f-ea8df0e7d74f None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:30:19.202393 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc45fc0-c488-4b29-bdf0-9b8bac57ae15 admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:30:19.202690 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc45fc0-c488-4b29-bdf0-9b8bac57ae15 admin admin] PATH_INFO: `/v3/roles` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:30:19.203243 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e16474-a951-474c-8b1f-ea8df0e7d74f None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:30:19.203595 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e16474-a951-474c-8b1f-ea8df0e7d74f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:30:19.205250 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc45fc0-c488-4b29-bdf0-9b8bac57ae15 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:30:19.206902 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc45fc0-c488-4b29-bdf0-9b8bac57ae15 admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:30:19.219745 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1236/2442] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 12:30:19 2022] GET /identity/v3/roles => generated 1420 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 15 12:30:19.223765 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc45fc0-c488-4b29-bdf0-9b8bac57ae15 admin admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:30:19.225752 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-4bc45fc0-c488-4b29-bdf0-9b8bac57ae15 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:30:19.236031 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-82e16474-a951-474c-8b1f-ea8df0e7d74f None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:30:19.238268 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-82e16474-a951-474c-8b1f-ea8df0e7d74f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:30:19.242971 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc45fc0-c488-4b29-bdf0-9b8bac57ae15 admin admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:30:19.249848 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1207/2443] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 12:30:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:30:19.250381 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f9bdec-b60b-45fe-b325-4767b666018c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:30:19.251060 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f9bdec-b60b-45fe-b325-4767b666018c admin admin] REQUEST_METHOD: `PUT` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:30:19.251341 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f9bdec-b60b-45fe-b325-4767b666018c admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:30:19.251650 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f9bdec-b60b-45fe-b325-4767b666018c admin admin] PATH_INFO: `/v3/projects/9a27904949a047e08aea23e5b60dcbc3/users/32d0c2cbec444dcfad46dece109c27bf/roles/9c769b5c82884766b2c510b1f0f1364c` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:30:19.254619 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f9bdec-b60b-45fe-b325-4767b666018c admin admin] RBAC: Authorizing `identity:create_grant(project_id=9a27904949a047e08aea23e5b60dcbc3, user_id=32d0c2cbec444dcfad46dece109c27bf, role_id=9c769b5c82884766b2c510b1f0f1364c)` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:30:19.256296 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e16474-a951-474c-8b1f-ea8df0e7d74f None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:30:19.262397 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f9bdec-b60b-45fe-b325-4767b666018c admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:30:19.266564 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e16474-a951-474c-8b1f-ea8df0e7d74f None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:30:19.271227 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d184287-caee-4e2b-91ac-a6bd7bcad916 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:30:19.271858 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d184287-caee-4e2b-91ac-a6bd7bcad916 admin admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:30:19.272168 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d184287-caee-4e2b-91ac-a6bd7bcad916 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:30:19.272443 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d184287-caee-4e2b-91ac-a6bd7bcad916 admin admin] PATH_INFO: `/v3/domains` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:30:19.273141 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d184287-caee-4e2b-91ac-a6bd7bcad916 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:30:19.273445 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d184287-caee-4e2b-91ac-a6bd7bcad916 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:30:19.275026 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d184287-caee-4e2b-91ac-a6bd7bcad916 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:30:19.277006 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1237/2444] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 15 12:30:19 2022] PUT /identity/v3/projects/9a27904949a047e08aea23e5b60dcbc3/users/32d0c2cbec444dcfad46dece109c27bf/roles/9c769b5c82884766b2c510b1f0f1364c => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:30:19.286623 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cb986d-a251-48a1-9a71-b353f73d953a None None] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:30:19.286912 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cb986d-a251-48a1-9a71-b353f73d953a None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:30:19.287204 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cb986d-a251-48a1-9a71-b353f73d953a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:30:19.288794 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1208/2445] 10.222.0.15 () {58 vars in 1153 bytes} [Thu Sep 15 12:30:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:30:19.293470 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d184287-caee-4e2b-91ac-a6bd7bcad916 admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:30:19.306339 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d184287-caee-4e2b-91ac-a6bd7bcad916 admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:30:19.327245 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.auth.core [None req-79cb986d-a251-48a1-9a71-b353f73d953a None None] MFA Rules not processed for user `32d0c2cbec444dcfad46dece109c27bf`. Rule list: `[]` (Enabled: `True`). {{(pid=81449) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:30:19.329065 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-79cb986d-a251-48a1-9a71-b353f73d953a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:30:19.390547 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f414a5f-18be-4192-acc3-a8c5b98e52e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:30:19.392460 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f414a5f-18be-4192-acc3-a8c5b98e52e0 admin admin] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:30:19.392940 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f414a5f-18be-4192-acc3-a8c5b98e52e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:30:19.393359 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f414a5f-18be-4192-acc3-a8c5b98e52e0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:30:19.393826 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f414a5f-18be-4192-acc3-a8c5b98e52e0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:30:19.395715 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f414a5f-18be-4192-acc3-a8c5b98e52e0 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:30:19.414755 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1238/2446] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 12:30:19 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 12:30:19.425752 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79cb986d-a251-48a1-9a71-b353f73d953a None None] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:30:19.428818 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1209/2447] 10.222.0.15 () {60 vars in 1153 bytes} [Thu Sep 15 12:30:19 2022] POST /identity/v3/projects => generated 357 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 12:30:19.428973 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-79cb986d-a251-48a1-9a71-b353f73d953a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:30:19.430377 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f414a5f-18be-4192-acc3-a8c5b98e52e0 admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:30:19.467749 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:30:19.496230 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79cb986d-a251-48a1-9a71-b353f73d953a None None] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:30:19.504059 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f414a5f-18be-4192-acc3-a8c5b98e52e0 admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:30:19.509335 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb38894c-6a77-47b1-9711-9e34be39d0aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:30:19.509953 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb38894c-6a77-47b1-9711-9e34be39d0aa admin admin] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:30:19.510225 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb38894c-6a77-47b1-9711-9e34be39d0aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:30:19.510494 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb38894c-6a77-47b1-9711-9e34be39d0aa admin admin] PATH_INFO: `/v3/users` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:30:19.511120 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb38894c-6a77-47b1-9711-9e34be39d0aa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:30:19.513066 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb38894c-6a77-47b1-9711-9e34be39d0aa admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:30:19.539731 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1210/2448] 10.222.0.15 () {60 vars in 1144 bytes} [Thu Sep 15 12:30:19 2022] POST /identity/v3/users => generated 279 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 15 12:30:19.540931 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb38894c-6a77-47b1-9711-9e34be39d0aa admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:30:19.551022 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb38894c-6a77-47b1-9711-9e34be39d0aa admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:30:19.552137 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc03994-540b-40a4-8f8e-5339031029a3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8191d3ccb874529b32f8d0026db629d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '540a32ef88764e8c8d9201ae22f6ddaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:30:19.552789 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc03994-540b-40a4-8f8e-5339031029a3 service cinder] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:30:19.553122 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc03994-540b-40a4-8f8e-5339031029a3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:30:19.553404 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc03994-540b-40a4-8f8e-5339031029a3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:30:19.555341 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-ecc03994-540b-40a4-8f8e-5339031029a3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:30:19.555655 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ad3c1c-e735-4d46-ae38-d66448cae288 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:30:19.556255 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ad3c1c-e735-4d46-ae38-d66448cae288 admin admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:30:19.556536 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ad3c1c-e735-4d46-ae38-d66448cae288 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:30:19.556847 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ad3c1c-e735-4d46-ae38-d66448cae288 admin admin] PATH_INFO: `/v3/roles` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:30:19.557483 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ad3c1c-e735-4d46-ae38-d66448cae288 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:30:19.559067 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ad3c1c-e735-4d46-ae38-d66448cae288 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:30:19.572583 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1211/2449] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 12:30:19 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 12:30:19.573609 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ad3c1c-e735-4d46-ae38-d66448cae288 admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:30:19.583132 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ad3c1c-e735-4d46-ae38-d66448cae288 admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:30:19.588310 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a51377d-7f7a-4768-9c1e-b6079128db00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:30:19.588935 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a51377d-7f7a-4768-9c1e-b6079128db00 admin admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:30:19.589241 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a51377d-7f7a-4768-9c1e-b6079128db00 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:30:19.589524 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a51377d-7f7a-4768-9c1e-b6079128db00 admin admin] PATH_INFO: `/v3/roles` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:30:19.590140 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a51377d-7f7a-4768-9c1e-b6079128db00 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:30:19.591674 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a51377d-7f7a-4768-9c1e-b6079128db00 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:30:19.601966 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1212/2450] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 12:30:19 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 12:30:19.605943 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a51377d-7f7a-4768-9c1e-b6079128db00 admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:30:19.615747 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a51377d-7f7a-4768-9c1e-b6079128db00 admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:30:19.619308 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc03994-540b-40a4-8f8e-5339031029a3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:30:19.620375 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fce6922-224c-4314-871b-da50254aff5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:30:19.620774 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc03994-540b-40a4-8f8e-5339031029a3 service cinder] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:30:19.621028 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fce6922-224c-4314-871b-da50254aff5c admin admin] REQUEST_METHOD: `PUT` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:30:19.621305 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fce6922-224c-4314-871b-da50254aff5c admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:30:19.621577 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fce6922-224c-4314-871b-da50254aff5c admin admin] PATH_INFO: `/v3/projects/cf9009a5a6e04fd4ba177ba444305a0b/users/052af2fdbbd243dab963deabf0d43f82/roles/0679d6628a25438d90dc4bdc0401eda2` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:30:19.632238 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1239/2451] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 12:30:19 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 207 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 15 12:30:19.639617 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fce6922-224c-4314-871b-da50254aff5c admin admin] RBAC: Authorizing `identity:create_grant(project_id=cf9009a5a6e04fd4ba177ba444305a0b, user_id=052af2fdbbd243dab963deabf0d43f82, role_id=0679d6628a25438d90dc4bdc0401eda2)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:30:19.642569 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fce6922-224c-4314-871b-da50254aff5c admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:30:19.651425 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:30:19.666407 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1213/2452] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 15 12:30:19 2022] PUT /identity/v3/projects/cf9009a5a6e04fd4ba177ba444305a0b/users/052af2fdbbd243dab963deabf0d43f82/roles/0679d6628a25438d90dc4bdc0401eda2 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 12:30:19.668294 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c779d61-de0d-4a94-9ba9-d9dcc4b441f5 None None] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:30:19.668583 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c779d61-de0d-4a94-9ba9-d9dcc4b441f5 None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:30:19.668896 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c779d61-de0d-4a94-9ba9-d9dcc4b441f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:30:19.698436 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.auth.core [None req-5c779d61-de0d-4a94-9ba9-d9dcc4b441f5 None None] MFA Rules not processed for user `052af2fdbbd243dab963deabf0d43f82`. Rule list: `[]` (Enabled: `True`). {{(pid=81449) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:30:19.700179 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-5c779d61-de0d-4a94-9ba9-d9dcc4b441f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:30:19.776887 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1240/2453] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 12:30:19 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:30:19.781316 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fce6922-224c-4314-871b-da50254aff5c admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:30:19.846826 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fce6922-224c-4314-871b-da50254aff5c admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:30:19.853224 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c64d92-aea3-4f17-a293-13bd39e2d149 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:30:19.853944 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c64d92-aea3-4f17-a293-13bd39e2d149 admin admin] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:30:19.854279 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c64d92-aea3-4f17-a293-13bd39e2d149 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:30:19.854639 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c64d92-aea3-4f17-a293-13bd39e2d149 admin admin] PATH_INFO: `/v3/projects` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:30:19.855377 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c64d92-aea3-4f17-a293-13bd39e2d149 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:30:19.857713 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c64d92-aea3-4f17-a293-13bd39e2d149 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:30:19.889669 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1214/2454] 10.222.0.15 () {60 vars in 1153 bytes} [Thu Sep 15 12:30:19 2022] POST /identity/v3/projects => generated 353 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 12:30:19.891748 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c779d61-de0d-4a94-9ba9-d9dcc4b441f5 None None] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:30:19.904331 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c779d61-de0d-4a94-9ba9-d9dcc4b441f5 None None] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:30:19.935171 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:30:19.976441 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348bb381-73fc-40e6-959c-93f5afe366f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:30:19.977305 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348bb381-73fc-40e6-959c-93f5afe366f6 admin admin] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:30:19.977580 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348bb381-73fc-40e6-959c-93f5afe366f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:30:19.977847 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348bb381-73fc-40e6-959c-93f5afe366f6 admin admin] PATH_INFO: `/v3/users` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:30:19.978482 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348bb381-73fc-40e6-959c-93f5afe366f6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:30:19.980450 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348bb381-73fc-40e6-959c-93f5afe366f6 admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:30:20.004167 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1241/2455] 10.222.0.15 () {60 vars in 1144 bytes} [Thu Sep 15 12:30:19 2022] POST /identity/v3/users => generated 283 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 12:30:20.008234 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c64d92-aea3-4f17-a293-13bd39e2d149 admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:30:20.019401 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c64d92-aea3-4f17-a293-13bd39e2d149 admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:30:20.023987 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f37551e-4b16-43e5-ae71-93e9cc5c8e9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:30:20.024589 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f37551e-4b16-43e5-ae71-93e9cc5c8e9b admin admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:30:20.024900 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f37551e-4b16-43e5-ae71-93e9cc5c8e9b admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:30:20.025175 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f37551e-4b16-43e5-ae71-93e9cc5c8e9b admin admin] PATH_INFO: `/v3/roles` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:30:20.025813 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f37551e-4b16-43e5-ae71-93e9cc5c8e9b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:30:20.027386 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f37551e-4b16-43e5-ae71-93e9cc5c8e9b admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:30:20.037746 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1215/2456] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 12:30:20 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 15 12:30:20.041746 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348bb381-73fc-40e6-959c-93f5afe366f6 admin admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:30:20.052106 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348bb381-73fc-40e6-959c-93f5afe366f6 admin admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:30:20.057050 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfc9355-7850-423b-b986-fa07e10f525d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:30:20.057710 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfc9355-7850-423b-b986-fa07e10f525d admin admin] REQUEST_METHOD: `PUT` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:30:20.057995 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfc9355-7850-423b-b986-fa07e10f525d admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:30:20.058299 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfc9355-7850-423b-b986-fa07e10f525d admin admin] PATH_INFO: `/v3/projects/9866729aa40c4f4c956a7c871cb50d76/users/b7460aa81af14ac48d8166afb90d351b/roles/9c769b5c82884766b2c510b1f0f1364c` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:30:20.074495 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cfc9355-7850-423b-b986-fa07e10f525d admin admin] RBAC: Authorizing `identity:create_grant(project_id=9866729aa40c4f4c956a7c871cb50d76, user_id=b7460aa81af14ac48d8166afb90d351b, role_id=9c769b5c82884766b2c510b1f0f1364c)` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:30:20.077305 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cfc9355-7850-423b-b986-fa07e10f525d admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:30:20.092259 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1242/2457] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 15 12:30:20 2022] PUT /identity/v3/projects/9866729aa40c4f4c956a7c871cb50d76/users/b7460aa81af14ac48d8166afb90d351b/roles/9c769b5c82884766b2c510b1f0f1364c => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:30:20.097027 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f37551e-4b16-43e5-ae71-93e9cc5c8e9b admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:30:20.106886 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f37551e-4b16-43e5-ae71-93e9cc5c8e9b admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:30:20.161915 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e640f9e8-6146-401a-b2cb-5891433c11a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:30:20.162505 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e640f9e8-6146-401a-b2cb-5891433c11a8 admin admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:30:20.162786 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e640f9e8-6146-401a-b2cb-5891433c11a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:30:20.163060 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e640f9e8-6146-401a-b2cb-5891433c11a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:30:20.163696 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e640f9e8-6146-401a-b2cb-5891433c11a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:30:20.165315 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e640f9e8-6146-401a-b2cb-5891433c11a8 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:30:20.175246 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1216/2458] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 12:30:20 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 12:30:20.179353 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfc9355-7850-423b-b986-fa07e10f525d admin admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:30:20.190112 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfc9355-7850-423b-b986-fa07e10f525d admin admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:30:20.195119 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32bedc1-3abd-43bb-9136-0308ea56cc4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:30:20.195885 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32bedc1-3abd-43bb-9136-0308ea56cc4d admin admin] REQUEST_METHOD: `PUT` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:30:20.196186 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32bedc1-3abd-43bb-9136-0308ea56cc4d admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:30:20.196505 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32bedc1-3abd-43bb-9136-0308ea56cc4d admin admin] PATH_INFO: `/v3/projects/9866729aa40c4f4c956a7c871cb50d76/users/b7460aa81af14ac48d8166afb90d351b/roles/9c769b5c82884766b2c510b1f0f1364c` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:30:20.198502 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32bedc1-3abd-43bb-9136-0308ea56cc4d admin admin] RBAC: Authorizing `identity:create_grant(project_id=9866729aa40c4f4c956a7c871cb50d76, user_id=b7460aa81af14ac48d8166afb90d351b, role_id=9c769b5c82884766b2c510b1f0f1364c)` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:30:20.201188 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32bedc1-3abd-43bb-9136-0308ea56cc4d admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:30:20.212086 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1243/2459] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 15 12:30:20 2022] PUT /identity/v3/projects/9866729aa40c4f4c956a7c871cb50d76/users/b7460aa81af14ac48d8166afb90d351b/roles/9c769b5c82884766b2c510b1f0f1364c => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:30:20.223176 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72c60c1-22b3-478f-b6fc-00827c3ed8c6 None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:30:20.223461 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72c60c1-22b3-478f-b6fc-00827c3ed8c6 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:30:20.223732 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72c60c1-22b3-478f-b6fc-00827c3ed8c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:30:20.252494 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-d72c60c1-22b3-478f-b6fc-00827c3ed8c6 None None] MFA Rules not processed for user `b7460aa81af14ac48d8166afb90d351b`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:30:20.253733 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-d72c60c1-22b3-478f-b6fc-00827c3ed8c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:30:20.314043 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:30:20.324090 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1217/2460] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 12:30:20 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 12:30:20.337279 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32bedc1-3abd-43bb-9136-0308ea56cc4d admin admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:30:20.397044 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32bedc1-3abd-43bb-9136-0308ea56cc4d admin admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:30:20.401136 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26bfad9-4824-46e6-a1f3-195efb151bd7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8191d3ccb874529b32f8d0026db629d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '540a32ef88764e8c8d9201ae22f6ddaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:30:20.401738 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26bfad9-4824-46e6-a1f3-195efb151bd7 service cinder] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:30:20.402012 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26bfad9-4824-46e6-a1f3-195efb151bd7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:30:20.402284 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26bfad9-4824-46e6-a1f3-195efb151bd7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:30:20.412825 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c26bfad9-4824-46e6-a1f3-195efb151bd7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:30:20.414228 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c26bfad9-4824-46e6-a1f3-195efb151bd7 service cinder] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:30:20.424816 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1244/2461] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 12:30:20 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 15 12:30:20.530026 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72c60c1-22b3-478f-b6fc-00827c3ed8c6 None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:30:20.541609 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72c60c1-22b3-478f-b6fc-00827c3ed8c6 None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:30:20.545257 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8911d426-da8c-44ea-a64d-d3f0778b8c19 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8191d3ccb874529b32f8d0026db629d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '540a32ef88764e8c8d9201ae22f6ddaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:30:20.545878 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8911d426-da8c-44ea-a64d-d3f0778b8c19 service cinder] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:30:20.546179 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8911d426-da8c-44ea-a64d-d3f0778b8c19 service cinder] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:30:20.546447 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8911d426-da8c-44ea-a64d-d3f0778b8c19 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:30:20.611071 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8911d426-da8c-44ea-a64d-d3f0778b8c19 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:30:20.612537 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8911d426-da8c-44ea-a64d-d3f0778b8c19 service cinder] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:30:20.622260 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1218/2462] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 12:30:20 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 12:30:24.306478 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26bfad9-4824-46e6-a1f3-195efb151bd7 service cinder] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:30:24.308120 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-c26bfad9-4824-46e6-a1f3-195efb151bd7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:30:24.326870 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26bfad9-4824-46e6-a1f3-195efb151bd7 service cinder] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:30:24.334264 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595e2f20-246d-470e-afef-c2ae62d6e176 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8191d3ccb874529b32f8d0026db629d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '540a32ef88764e8c8d9201ae22f6ddaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:30:24.335546 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595e2f20-246d-470e-afef-c2ae62d6e176 service cinder] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:30:24.335964 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595e2f20-246d-470e-afef-c2ae62d6e176 service cinder] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:30:24.336887 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595e2f20-246d-470e-afef-c2ae62d6e176 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:30:24.455991 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595e2f20-246d-470e-afef-c2ae62d6e176 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:30:24.457466 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595e2f20-246d-470e-afef-c2ae62d6e176 service cinder] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:30:24.469119 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1245/2463] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 12:30:24 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 12:30:24.990146 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa7582a-9db8-438f-8b63-d9e1e65625a3 None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:30:24.990591 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa7582a-9db8-438f-8b63-d9e1e65625a3 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:30:24.990779 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa7582a-9db8-438f-8b63-d9e1e65625a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:30:25.010630 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-8fa7582a-9db8-438f-8b63-d9e1e65625a3 None None] MFA Rules not processed for user `0818d1847d5e42b6a942557f95b09d05`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:30:25.012049 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-8fa7582a-9db8-438f-8b63-d9e1e65625a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:30:25.066124 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1219/2464] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 12:30:24 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:30:25.297847 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595e2f20-246d-470e-afef-c2ae62d6e176 service cinder] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:30:25.299401 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-595e2f20-246d-470e-afef-c2ae62d6e176 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:30:25.321419 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:30:25.358609 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595e2f20-246d-470e-afef-c2ae62d6e176 service cinder] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:30:25.362651 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe41f70-db94-4d50-bd10-fdac3f322128 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfbb4d6809064f9cb9e06626ed7d6814', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '540a32ef88764e8c8d9201ae22f6ddaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:30:25.363248 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe41f70-db94-4d50-bd10-fdac3f322128 service neutron] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:30:25.363529 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe41f70-db94-4d50-bd10-fdac3f322128 service neutron] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:30:25.363795 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe41f70-db94-4d50-bd10-fdac3f322128 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:30:25.365477 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-abe41f70-db94-4d50-bd10-fdac3f322128 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:30:25.374710 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe41f70-db94-4d50-bd10-fdac3f322128 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:30:25.376521 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abe41f70-db94-4d50-bd10-fdac3f322128 service neutron] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:30:25.384484 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1246/2465] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 12:30:25 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 12:30:35.059478 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=55 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:30:35.380919 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=17 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:30:41.021888 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa7582a-9db8-438f-8b63-d9e1e65625a3 None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:30:41.127157 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa7582a-9db8-438f-8b63-d9e1e65625a3 None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:30:41.132960 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25b7117-2007-4960-96d1-af57340c5b93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:30:41.133725 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25b7117-2007-4960-96d1-af57340c5b93 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:30:41.134004 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25b7117-2007-4960-96d1-af57340c5b93 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:30:41.134274 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25b7117-2007-4960-96d1-af57340c5b93 admin admin] PATH_INFO: `/v3/users/052af2fdbbd243dab963deabf0d43f82` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:30:41.136118 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25b7117-2007-4960-96d1-af57340c5b93 admin admin] RBAC: Authorizing `identity:delete_user(user_id=052af2fdbbd243dab963deabf0d43f82)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:30:41.138134 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25b7117-2007-4960-96d1-af57340c5b93 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:30:41.193978 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-f25b7117-2007-4960-96d1-af57340c5b93 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '052af2fdbbd243dab963deabf0d43f82'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:30:41.208856 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-f25b7117-2007-4960-96d1-af57340c5b93 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '052af2fdbbd243dab963deabf0d43f82'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:30:41.218260 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-f25b7117-2007-4960-96d1-af57340c5b93 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '052af2fdbbd243dab963deabf0d43f82'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:30:41.219205 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-f25b7117-2007-4960-96d1-af57340c5b93 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '052af2fdbbd243dab963deabf0d43f82'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:30:41.229485 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-f25b7117-2007-4960-96d1-af57340c5b93 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '052af2fdbbd243dab963deabf0d43f82'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:30:41.241607 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04b3638-e51a-4bb2-9d6c-473f254d663d None None] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:30:41.242095 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04b3638-e51a-4bb2-9d6c-473f254d663d None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:30:41.242095 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04b3638-e51a-4bb2-9d6c-473f254d663d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:30:41.244884 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1220/2466] 10.222.0.15 () {58 vars in 1224 bytes} [Thu Sep 15 12:30:41 2022] DELETE /identity/v3/users/052af2fdbbd243dab963deabf0d43f82 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:30:41.271331 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.auth.core [None req-b04b3638-e51a-4bb2-9d6c-473f254d663d None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81449) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:30:41.272551 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-b04b3638-e51a-4bb2-9d6c-473f254d663d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:30:41.343958 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1247/2467] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 12:30:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:30:41.357996 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25b7117-2007-4960-96d1-af57340c5b93 admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:30:41.361337 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-f25b7117-2007-4960-96d1-af57340c5b93 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:30:41.440087 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25b7117-2007-4960-96d1-af57340c5b93 admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:30:41.447031 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d43496-1e78-434a-9dae-dbf2c0023724 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfbb4d6809064f9cb9e06626ed7d6814', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '540a32ef88764e8c8d9201ae22f6ddaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:30:41.448377 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d43496-1e78-434a-9dae-dbf2c0023724 service neutron] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:30:41.448743 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d43496-1e78-434a-9dae-dbf2c0023724 service neutron] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:30:41.449258 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d43496-1e78-434a-9dae-dbf2c0023724 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:30:41.467355 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d43496-1e78-434a-9dae-dbf2c0023724 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:30:41.469046 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d43496-1e78-434a-9dae-dbf2c0023724 service neutron] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:30:41.484954 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1221/2468] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 12:30:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 12:30:41.879215 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04b3638-e51a-4bb2-9d6c-473f254d663d None None] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:30:41.880989 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-b04b3638-e51a-4bb2-9d6c-473f254d663d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:30:41.895015 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04b3638-e51a-4bb2-9d6c-473f254d663d None None] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:30:41.901399 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0219af62-88e2-4f36-97b3-e7ed48b4f5ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:30:41.902196 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0219af62-88e2-4f36-97b3-e7ed48b4f5ea admin admin] REQUEST_METHOD: `DELETE` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:30:41.913111 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0219af62-88e2-4f36-97b3-e7ed48b4f5ea admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:30:41.913573 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0219af62-88e2-4f36-97b3-e7ed48b4f5ea admin admin] PATH_INFO: `/v3/projects/cf9009a5a6e04fd4ba177ba444305a0b` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:30:41.915124 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0219af62-88e2-4f36-97b3-e7ed48b4f5ea admin admin] RBAC: Authorizing `identity:delete_project(project_id=cf9009a5a6e04fd4ba177ba444305a0b)` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:30:41.917108 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0219af62-88e2-4f36-97b3-e7ed48b4f5ea admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:30:41.977392 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-0219af62-88e2-4f36-97b3-e7ed48b4f5ea admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cf9009a5a6e04fd4ba177ba444305a0b', 'request_id': 'req-0219af62-88e2-4f36-97b3-e7ed48b4f5ea'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:30:41.985624 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-0219af62-88e2-4f36-97b3-e7ed48b4f5ea admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cf9009a5a6e04fd4ba177ba444305a0b', 'request_id': 'req-0219af62-88e2-4f36-97b3-e7ed48b4f5ea'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:30:41.992167 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-0219af62-88e2-4f36-97b3-e7ed48b4f5ea admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cf9009a5a6e04fd4ba177ba444305a0b', 'request_id': 'req-0219af62-88e2-4f36-97b3-e7ed48b4f5ea'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:30:42.003725 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-0219af62-88e2-4f36-97b3-e7ed48b4f5ea admin admin] The token cache is being invalidate because project cf9009a5a6e04fd4ba177ba444305a0b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81449) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 12:30:42.004015 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-0219af62-88e2-4f36-97b3-e7ed48b4f5ea admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:30:42.004677 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-0219af62-88e2-4f36-97b3-e7ed48b4f5ea admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:30:42.006438 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1248/2469] 10.222.0.15 () {58 vars in 1233 bytes} [Thu Sep 15 12:30:41 2022] DELETE /identity/v3/projects/cf9009a5a6e04fd4ba177ba444305a0b => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:30:42.010216 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d43496-1e78-434a-9dae-dbf2c0023724 service neutron] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:30:42.020518 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d43496-1e78-434a-9dae-dbf2c0023724 service neutron] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:30:42.040292 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:30:42.086786 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b1f1c2-3a68-4343-a58a-59eb904c15a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:30:42.087460 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b1f1c2-3a68-4343-a58a-59eb904c15a0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:30:42.087741 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b1f1c2-3a68-4343-a58a-59eb904c15a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:30:42.088094 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b1f1c2-3a68-4343-a58a-59eb904c15a0 admin admin] PATH_INFO: `/v3/users/b7460aa81af14ac48d8166afb90d351b` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:30:42.094600 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b1f1c2-3a68-4343-a58a-59eb904c15a0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b7460aa81af14ac48d8166afb90d351b)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:30:42.096509 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b1f1c2-3a68-4343-a58a-59eb904c15a0 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:30:42.149017 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-b1b1f1c2-3a68-4343-a58a-59eb904c15a0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b7460aa81af14ac48d8166afb90d351b'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:30:42.166385 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-b1b1f1c2-3a68-4343-a58a-59eb904c15a0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b7460aa81af14ac48d8166afb90d351b'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:30:42.174876 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-b1b1f1c2-3a68-4343-a58a-59eb904c15a0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b7460aa81af14ac48d8166afb90d351b'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:30:42.176283 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-b1b1f1c2-3a68-4343-a58a-59eb904c15a0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b7460aa81af14ac48d8166afb90d351b'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:30:42.184213 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-b1b1f1c2-3a68-4343-a58a-59eb904c15a0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b7460aa81af14ac48d8166afb90d351b'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:30:42.191530 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1222/2470] 10.222.0.15 () {58 vars in 1224 bytes} [Thu Sep 15 12:30:42 2022] DELETE /identity/v3/users/b7460aa81af14ac48d8166afb90d351b => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:30:42.717809 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0219af62-88e2-4f36-97b3-e7ed48b4f5ea admin admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:30:42.720557 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-0219af62-88e2-4f36-97b3-e7ed48b4f5ea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:30:42.736038 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0219af62-88e2-4f36-97b3-e7ed48b4f5ea admin admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:30:42.807055 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c41796b-1699-47f6-99ce-2f19177fd9e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:30:42.807635 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c41796b-1699-47f6-99ce-2f19177fd9e7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:30:42.807913 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c41796b-1699-47f6-99ce-2f19177fd9e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:30:42.808182 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c41796b-1699-47f6-99ce-2f19177fd9e7 admin admin] PATH_INFO: `/v3/projects/9866729aa40c4f4c956a7c871cb50d76` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:30:42.809341 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c41796b-1699-47f6-99ce-2f19177fd9e7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9866729aa40c4f4c956a7c871cb50d76)` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:30:42.811277 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c41796b-1699-47f6-99ce-2f19177fd9e7 admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:30:42.843825 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:30:42.863886 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-1c41796b-1699-47f6-99ce-2f19177fd9e7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9866729aa40c4f4c956a7c871cb50d76', 'request_id': 'req-1c41796b-1699-47f6-99ce-2f19177fd9e7'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:30:42.872864 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-1c41796b-1699-47f6-99ce-2f19177fd9e7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9866729aa40c4f4c956a7c871cb50d76', 'request_id': 'req-1c41796b-1699-47f6-99ce-2f19177fd9e7'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:30:42.879200 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-1c41796b-1699-47f6-99ce-2f19177fd9e7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9866729aa40c4f4c956a7c871cb50d76', 'request_id': 'req-1c41796b-1699-47f6-99ce-2f19177fd9e7'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:30:42.890804 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-1c41796b-1699-47f6-99ce-2f19177fd9e7 admin admin] The token cache is being invalidate because project 9866729aa40c4f4c956a7c871cb50d76 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81449) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 12:30:42.891117 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-1c41796b-1699-47f6-99ce-2f19177fd9e7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:30:42.891774 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-1c41796b-1699-47f6-99ce-2f19177fd9e7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:30:42.893269 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1249/2471] 10.222.0.15 () {58 vars in 1233 bytes} [Thu Sep 15 12:30:42 2022] DELETE /identity/v3/projects/9866729aa40c4f4c956a7c871cb50d76 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:30:42.899362 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41132de4-50d9-4f6b-a612-bc6b741023a2 None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:30:42.899728 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41132de4-50d9-4f6b-a612-bc6b741023a2 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:30:42.899999 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41132de4-50d9-4f6b-a612-bc6b741023a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:30:42.925095 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-41132de4-50d9-4f6b-a612-bc6b741023a2 None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:30:42.927029 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-41132de4-50d9-4f6b-a612-bc6b741023a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:30:42.994264 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1223/2472] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 12:30:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:30:43.001878 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abf7758-386b-4477-b468-b0a4016fa55c None None] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:30:43.002231 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abf7758-386b-4477-b468-b0a4016fa55c None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:30:43.002576 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abf7758-386b-4477-b468-b0a4016fa55c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:30:43.025422 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.auth.core [None req-5abf7758-386b-4477-b468-b0a4016fa55c None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81449) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:30:43.026645 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-5abf7758-386b-4477-b468-b0a4016fa55c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:30:43.039211 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1250/2473] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 12:30:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 12:30:43.043398 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41132de4-50d9-4f6b-a612-bc6b741023a2 None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:30:43.053631 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41132de4-50d9-4f6b-a612-bc6b741023a2 None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:30:43.058386 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7d486c-1607-4fa6-8d15-06ca56169b5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:30:43.058939 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7d486c-1607-4fa6-8d15-06ca56169b5f admin admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:30:43.059213 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7d486c-1607-4fa6-8d15-06ca56169b5f admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:30:43.059506 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7d486c-1607-4fa6-8d15-06ca56169b5f admin admin] PATH_INFO: `/v3/domains` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:30:43.060140 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b7d486c-1607-4fa6-8d15-06ca56169b5f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:30:43.060438 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b7d486c-1607-4fa6-8d15-06ca56169b5f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:30:43.062050 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b7d486c-1607-4fa6-8d15-06ca56169b5f admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:30:43.074789 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1224/2474] 10.222.0.15 () {58 vars in 1153 bytes} [Thu Sep 15 12:30:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:30:43.079164 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5abf7758-386b-4477-b468-b0a4016fa55c None None] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:30:43.088977 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5abf7758-386b-4477-b468-b0a4016fa55c None None] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:30:43.093956 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24739d1-2a67-4780-b7b2-a2949256814d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:30:43.094561 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24739d1-2a67-4780-b7b2-a2949256814d admin admin] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:30:43.094841 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24739d1-2a67-4780-b7b2-a2949256814d admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:30:43.095115 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24739d1-2a67-4780-b7b2-a2949256814d admin admin] PATH_INFO: `/v3/projects` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:30:43.095745 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24739d1-2a67-4780-b7b2-a2949256814d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:30:43.098183 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24739d1-2a67-4780-b7b2-a2949256814d admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:30:43.126263 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1251/2475] 10.222.0.15 () {60 vars in 1153 bytes} [Thu Sep 15 12:30:43 2022] POST /identity/v3/projects => generated 387 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 12:30:43.130529 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7d486c-1607-4fa6-8d15-06ca56169b5f admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:30:43.142575 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7d486c-1607-4fa6-8d15-06ca56169b5f admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:30:43.169333 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:30:43.202597 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ef08cb-3410-4f37-9538-5db8bb418654 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:30:43.203225 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ef08cb-3410-4f37-9538-5db8bb418654 admin admin] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:30:43.203507 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ef08cb-3410-4f37-9538-5db8bb418654 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:30:43.203782 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ef08cb-3410-4f37-9538-5db8bb418654 admin admin] PATH_INFO: `/v3/users` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:30:43.204430 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ef08cb-3410-4f37-9538-5db8bb418654 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:30:43.206407 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ef08cb-3410-4f37-9538-5db8bb418654 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:30:43.229728 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1225/2476] 10.222.0.15 () {60 vars in 1144 bytes} [Thu Sep 15 12:30:43 2022] POST /identity/v3/users => generated 294 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 12:30:43.234030 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24739d1-2a67-4780-b7b2-a2949256814d admin admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:30:43.245127 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24739d1-2a67-4780-b7b2-a2949256814d admin admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:30:43.249928 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e41c04-2576-4b3b-bd9a-0d25d9d0c359 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:30:43.250529 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e41c04-2576-4b3b-bd9a-0d25d9d0c359 admin admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:30:43.250807 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e41c04-2576-4b3b-bd9a-0d25d9d0c359 admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:30:43.251078 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e41c04-2576-4b3b-bd9a-0d25d9d0c359 admin admin] PATH_INFO: `/v3/roles` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:30:43.251720 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e41c04-2576-4b3b-bd9a-0d25d9d0c359 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:30:43.253378 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e41c04-2576-4b3b-bd9a-0d25d9d0c359 admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:30:43.263824 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1252/2477] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 12:30:43 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 12:30:43.268084 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ef08cb-3410-4f37-9538-5db8bb418654 admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:30:43.278522 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ef08cb-3410-4f37-9538-5db8bb418654 admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:30:43.283402 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64fbf1b0-eaec-4d23-80c1-f7d26d2f9e82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:30:43.284012 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fbf1b0-eaec-4d23-80c1-f7d26d2f9e82 admin admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:30:43.284288 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fbf1b0-eaec-4d23-80c1-f7d26d2f9e82 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:30:43.285212 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fbf1b0-eaec-4d23-80c1-f7d26d2f9e82 admin admin] PATH_INFO: `/v3/roles` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:30:43.285851 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64fbf1b0-eaec-4d23-80c1-f7d26d2f9e82 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:30:43.287457 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64fbf1b0-eaec-4d23-80c1-f7d26d2f9e82 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:30:43.298461 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1226/2478] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 12:30:43 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 12:30:43.303138 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e41c04-2576-4b3b-bd9a-0d25d9d0c359 admin admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:30:43.313340 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e41c04-2576-4b3b-bd9a-0d25d9d0c359 admin admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:30:43.318221 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d03b0d-3e6c-4327-9224-fee7167a8698 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:30:43.318900 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d03b0d-3e6c-4327-9224-fee7167a8698 admin admin] REQUEST_METHOD: `PUT` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:30:43.319179 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d03b0d-3e6c-4327-9224-fee7167a8698 admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:30:43.319447 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d03b0d-3e6c-4327-9224-fee7167a8698 admin admin] PATH_INFO: `/v3/projects/e6b1ebf72aa14eda9e492c6375626f97/users/c50bb926f5fb44abad5bd52183b9538b/roles/0679d6628a25438d90dc4bdc0401eda2` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:30:43.334119 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d03b0d-3e6c-4327-9224-fee7167a8698 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e6b1ebf72aa14eda9e492c6375626f97, user_id=c50bb926f5fb44abad5bd52183b9538b, role_id=0679d6628a25438d90dc4bdc0401eda2)` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:30:43.337115 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d03b0d-3e6c-4327-9224-fee7167a8698 admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:30:43.350986 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1253/2479] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 15 12:30:43 2022] PUT /identity/v3/projects/e6b1ebf72aa14eda9e492c6375626f97/users/c50bb926f5fb44abad5bd52183b9538b/roles/0679d6628a25438d90dc4bdc0401eda2 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:30:43.361454 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d301db3-169c-4d87-8bfc-722bead9b17c None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:30:43.361736 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d301db3-169c-4d87-8bfc-722bead9b17c None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:30:43.362004 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d301db3-169c-4d87-8bfc-722bead9b17c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:30:43.391084 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-2d301db3-169c-4d87-8bfc-722bead9b17c None None] MFA Rules not processed for user `c50bb926f5fb44abad5bd52183b9538b`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:30:43.392278 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-2d301db3-169c-4d87-8bfc-722bead9b17c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:30:43.459297 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1227/2480] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 12:30:43 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:30:43.464157 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d03b0d-3e6c-4327-9224-fee7167a8698 admin admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:30:43.527740 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d03b0d-3e6c-4327-9224-fee7167a8698 admin admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:30:43.533188 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b014713-e080-4116-888e-7962a70cabd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:30:43.533798 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b014713-e080-4116-888e-7962a70cabd1 admin admin] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:30:43.534080 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b014713-e080-4116-888e-7962a70cabd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:30:43.534352 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b014713-e080-4116-888e-7962a70cabd1 admin admin] PATH_INFO: `/v3/projects` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:30:43.535590 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b014713-e080-4116-888e-7962a70cabd1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:30:43.537536 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b014713-e080-4116-888e-7962a70cabd1 admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:30:43.567851 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1254/2481] 10.222.0.15 () {60 vars in 1153 bytes} [Thu Sep 15 12:30:43 2022] POST /identity/v3/projects => generated 387 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 12:30:43.569227 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d301db3-169c-4d87-8bfc-722bead9b17c None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:30:43.580623 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d301db3-169c-4d87-8bfc-722bead9b17c None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:30:43.638810 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51c548b-54f6-403d-a0dd-3f44a3ca1474 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:30:43.639459 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51c548b-54f6-403d-a0dd-3f44a3ca1474 admin admin] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:30:43.639867 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51c548b-54f6-403d-a0dd-3f44a3ca1474 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:30:43.640147 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51c548b-54f6-403d-a0dd-3f44a3ca1474 admin admin] PATH_INFO: `/v3/users` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:30:43.640835 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51c548b-54f6-403d-a0dd-3f44a3ca1474 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:30:43.642795 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51c548b-54f6-403d-a0dd-3f44a3ca1474 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:30:43.667801 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1228/2482] 10.222.0.15 () {60 vars in 1144 bytes} [Thu Sep 15 12:30:43 2022] POST /identity/v3/users => generated 294 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 15 12:30:43.671792 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b014713-e080-4116-888e-7962a70cabd1 admin admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:30:43.684331 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b014713-e080-4116-888e-7962a70cabd1 admin admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:30:43.689480 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46e13e6-cbf5-46fd-8276-3195fbd09b67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:30:43.690098 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46e13e6-cbf5-46fd-8276-3195fbd09b67 admin admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:30:43.690428 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46e13e6-cbf5-46fd-8276-3195fbd09b67 admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:30:43.691356 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46e13e6-cbf5-46fd-8276-3195fbd09b67 admin admin] PATH_INFO: `/v3/roles` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:30:43.692096 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46e13e6-cbf5-46fd-8276-3195fbd09b67 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:30:43.693769 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46e13e6-cbf5-46fd-8276-3195fbd09b67 admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:30:43.709380 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1255/2483] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 12:30:43 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 12:30:43.709588 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51c548b-54f6-403d-a0dd-3f44a3ca1474 admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:30:43.720385 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51c548b-54f6-403d-a0dd-3f44a3ca1474 admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:30:43.725312 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3858833-f11b-47d6-9c2e-6878dc121299 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:30:43.725974 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3858833-f11b-47d6-9c2e-6878dc121299 admin admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:30:43.726259 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3858833-f11b-47d6-9c2e-6878dc121299 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:30:43.726528 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3858833-f11b-47d6-9c2e-6878dc121299 admin admin] PATH_INFO: `/v3/roles` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:30:43.727169 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3858833-f11b-47d6-9c2e-6878dc121299 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:30:43.728987 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3858833-f11b-47d6-9c2e-6878dc121299 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:30:43.740587 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1229/2484] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 12:30:43 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 15 12:30:43.744819 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46e13e6-cbf5-46fd-8276-3195fbd09b67 admin admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:30:43.755888 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46e13e6-cbf5-46fd-8276-3195fbd09b67 admin admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:30:43.760488 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6633a7d9-5e09-43e5-a85e-63cbd871ac6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:30:43.761173 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6633a7d9-5e09-43e5-a85e-63cbd871ac6b admin admin] REQUEST_METHOD: `PUT` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:30:43.761518 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6633a7d9-5e09-43e5-a85e-63cbd871ac6b admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:30:43.761714 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6633a7d9-5e09-43e5-a85e-63cbd871ac6b admin admin] PATH_INFO: `/v3/projects/a2a4a011f99c432c9144184d08c15e30/users/f2eaab9e71da431d9edfddad27095421/roles/0679d6628a25438d90dc4bdc0401eda2` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:30:43.776947 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6633a7d9-5e09-43e5-a85e-63cbd871ac6b admin admin] RBAC: Authorizing `identity:create_grant(project_id=a2a4a011f99c432c9144184d08c15e30, user_id=f2eaab9e71da431d9edfddad27095421, role_id=0679d6628a25438d90dc4bdc0401eda2)` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:30:43.779819 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6633a7d9-5e09-43e5-a85e-63cbd871ac6b admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:30:43.790507 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:DELETE=6,keystone:SELECT=89,keystone:INSERT=5 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:30:43.808829 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1256/2485] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 15 12:30:43 2022] PUT /identity/v3/projects/a2a4a011f99c432c9144184d08c15e30/users/f2eaab9e71da431d9edfddad27095421/roles/0679d6628a25438d90dc4bdc0401eda2 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 12:30:43.810890 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de27bb2-93f5-4255-9277-1f3be24c15bb None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:30:43.811212 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de27bb2-93f5-4255-9277-1f3be24c15bb None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:30:43.811492 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de27bb2-93f5-4255-9277-1f3be24c15bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:30:43.840740 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-6de27bb2-93f5-4255-9277-1f3be24c15bb None None] MFA Rules not processed for user `f2eaab9e71da431d9edfddad27095421`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:30:43.841886 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-6de27bb2-93f5-4255-9277-1f3be24c15bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:30:43.845809 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:30:43.908367 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1230/2486] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 12:30:43 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:30:43.913495 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6633a7d9-5e09-43e5-a85e-63cbd871ac6b admin admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:30:43.980948 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6633a7d9-5e09-43e5-a85e-63cbd871ac6b admin admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:30:43.985681 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf0a7ae-b0d2-4619-8284-3d515808a110 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:30:43.986295 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf0a7ae-b0d2-4619-8284-3d515808a110 admin admin] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:30:43.986584 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf0a7ae-b0d2-4619-8284-3d515808a110 admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:30:43.986861 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf0a7ae-b0d2-4619-8284-3d515808a110 admin admin] PATH_INFO: `/v3/projects` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:30:43.987513 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf0a7ae-b0d2-4619-8284-3d515808a110 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:30:43.990102 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf0a7ae-b0d2-4619-8284-3d515808a110 admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:30:44.017173 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1257/2487] 10.222.0.15 () {60 vars in 1153 bytes} [Thu Sep 15 12:30:43 2022] POST /identity/v3/projects => generated 385 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 15 12:30:44.021383 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de27bb2-93f5-4255-9277-1f3be24c15bb None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:30:44.032288 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de27bb2-93f5-4255-9277-1f3be24c15bb None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:30:44.093321 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad851c0-01a2-44fe-9f29-0814944346bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:30:44.093908 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad851c0-01a2-44fe-9f29-0814944346bf admin admin] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:30:44.094148 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad851c0-01a2-44fe-9f29-0814944346bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:30:44.094416 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad851c0-01a2-44fe-9f29-0814944346bf admin admin] PATH_INFO: `/v3/users` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:30:44.095047 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad851c0-01a2-44fe-9f29-0814944346bf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:30:44.097674 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad851c0-01a2-44fe-9f29-0814944346bf admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:30:44.123313 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1231/2488] 10.222.0.15 () {60 vars in 1144 bytes} [Thu Sep 15 12:30:44 2022] POST /identity/v3/users => generated 299 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 12:30:44.124557 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf0a7ae-b0d2-4619-8284-3d515808a110 admin admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:30:44.135258 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf0a7ae-b0d2-4619-8284-3d515808a110 admin admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:30:44.140004 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454addf8-0810-434f-bd08-2fec756f3081 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:30:44.140636 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454addf8-0810-434f-bd08-2fec756f3081 admin admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:30:44.140963 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454addf8-0810-434f-bd08-2fec756f3081 admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:30:44.141622 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454addf8-0810-434f-bd08-2fec756f3081 admin admin] PATH_INFO: `/v3/roles` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:30:44.142324 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454addf8-0810-434f-bd08-2fec756f3081 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:30:44.143972 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454addf8-0810-434f-bd08-2fec756f3081 admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:30:44.155913 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1258/2489] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 12:30:44 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 12:30:44.160646 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad851c0-01a2-44fe-9f29-0814944346bf admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:30:44.170522 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad851c0-01a2-44fe-9f29-0814944346bf admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:30:44.176114 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e510e0c-6b5e-4472-a8bc-844a2350ff6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:30:44.176842 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e510e0c-6b5e-4472-a8bc-844a2350ff6e admin admin] REQUEST_METHOD: `PUT` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:30:44.177121 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e510e0c-6b5e-4472-a8bc-844a2350ff6e admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:30:44.177392 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e510e0c-6b5e-4472-a8bc-844a2350ff6e admin admin] PATH_INFO: `/v3/projects/ba700df8e55b4753a23c876d06fd042a/users/2afa74bb7718438fa45e697e99c349e7/roles/9c769b5c82884766b2c510b1f0f1364c` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:30:44.196799 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e510e0c-6b5e-4472-a8bc-844a2350ff6e admin admin] RBAC: Authorizing `identity:create_grant(project_id=ba700df8e55b4753a23c876d06fd042a, user_id=2afa74bb7718438fa45e697e99c349e7, role_id=9c769b5c82884766b2c510b1f0f1364c)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:30:44.197270 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e510e0c-6b5e-4472-a8bc-844a2350ff6e admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:30:44.211834 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1232/2490] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 15 12:30:44 2022] PUT /identity/v3/projects/ba700df8e55b4753a23c876d06fd042a/users/2afa74bb7718438fa45e697e99c349e7/roles/9c769b5c82884766b2c510b1f0f1364c => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 12:30:44.215844 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454addf8-0810-434f-bd08-2fec756f3081 admin admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:30:44.227218 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454addf8-0810-434f-bd08-2fec756f3081 admin admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:30:44.285516 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9938b75-0902-4336-a634-2fd0c6de86f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:30:44.286160 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9938b75-0902-4336-a634-2fd0c6de86f1 admin admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:30:44.286442 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9938b75-0902-4336-a634-2fd0c6de86f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:30:44.286706 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9938b75-0902-4336-a634-2fd0c6de86f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:30:44.287343 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9938b75-0902-4336-a634-2fd0c6de86f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:30:44.288941 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9938b75-0902-4336-a634-2fd0c6de86f1 admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:30:44.299196 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1259/2491] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 12:30:44 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 12:30:44.303459 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e510e0c-6b5e-4472-a8bc-844a2350ff6e admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:30:44.313838 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e510e0c-6b5e-4472-a8bc-844a2350ff6e admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:30:44.318569 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418ddf27-9607-43fd-a8f7-aecd2bd59839 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:30:44.319222 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418ddf27-9607-43fd-a8f7-aecd2bd59839 admin admin] REQUEST_METHOD: `PUT` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:30:44.319504 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418ddf27-9607-43fd-a8f7-aecd2bd59839 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:30:44.319776 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418ddf27-9607-43fd-a8f7-aecd2bd59839 admin admin] PATH_INFO: `/v3/projects/ba700df8e55b4753a23c876d06fd042a/users/2afa74bb7718438fa45e697e99c349e7/roles/9c769b5c82884766b2c510b1f0f1364c` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:30:44.321719 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418ddf27-9607-43fd-a8f7-aecd2bd59839 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ba700df8e55b4753a23c876d06fd042a, user_id=2afa74bb7718438fa45e697e99c349e7, role_id=9c769b5c82884766b2c510b1f0f1364c)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:30:44.324332 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418ddf27-9607-43fd-a8f7-aecd2bd59839 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:30:44.335563 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1233/2492] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 15 12:30:44 2022] PUT /identity/v3/projects/ba700df8e55b4753a23c876d06fd042a/users/2afa74bb7718438fa45e697e99c349e7/roles/9c769b5c82884766b2c510b1f0f1364c => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 12:30:44.346110 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7939296-a223-4015-ba9e-50051cc012cc None None] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:30:44.346337 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7939296-a223-4015-ba9e-50051cc012cc None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:30:44.346649 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7939296-a223-4015-ba9e-50051cc012cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:30:44.375454 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.auth.core [None req-a7939296-a223-4015-ba9e-50051cc012cc None None] MFA Rules not processed for user `2afa74bb7718438fa45e697e99c349e7`. Rule list: `[]` (Enabled: `True`). {{(pid=81449) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:30:44.376647 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-a7939296-a223-4015-ba9e-50051cc012cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:30:44.437966 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:30:44.448196 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1260/2493] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 12:30:44 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 12:30:44.459535 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418ddf27-9607-43fd-a8f7-aecd2bd59839 admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:30:44.461640 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-418ddf27-9607-43fd-a8f7-aecd2bd59839 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:30:44.520733 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418ddf27-9607-43fd-a8f7-aecd2bd59839 admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:30:44.524350 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91579eba-e8b8-410b-9344-79889b440f5d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8191d3ccb874529b32f8d0026db629d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '540a32ef88764e8c8d9201ae22f6ddaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:30:44.525069 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91579eba-e8b8-410b-9344-79889b440f5d service cinder] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:30:44.525348 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91579eba-e8b8-410b-9344-79889b440f5d service cinder] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:30:44.525622 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91579eba-e8b8-410b-9344-79889b440f5d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:30:44.535781 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91579eba-e8b8-410b-9344-79889b440f5d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:30:44.537247 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91579eba-e8b8-410b-9344-79889b440f5d service cinder] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:30:44.542455 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:30:44.547808 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1234/2494] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 12:30:44 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 12:30:45.231941 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7939296-a223-4015-ba9e-50051cc012cc None None] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:30:45.238930 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-a7939296-a223-4015-ba9e-50051cc012cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:30:45.258223 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7939296-a223-4015-ba9e-50051cc012cc None None] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:30:45.263247 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f80d93-7f43-4023-a499-e0cf48cee186 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8191d3ccb874529b32f8d0026db629d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '540a32ef88764e8c8d9201ae22f6ddaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:30:45.264685 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f80d93-7f43-4023-a499-e0cf48cee186 service cinder] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:30:45.265436 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f80d93-7f43-4023-a499-e0cf48cee186 service cinder] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:30:45.266651 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f80d93-7f43-4023-a499-e0cf48cee186 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:30:45.395866 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f80d93-7f43-4023-a499-e0cf48cee186 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:30:45.402269 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f80d93-7f43-4023-a499-e0cf48cee186 service cinder] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:30:45.426482 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1261/2495] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 12:30:45 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 197 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 12:30:46.568898 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91579eba-e8b8-410b-9344-79889b440f5d service cinder] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:30:46.570723 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-91579eba-e8b8-410b-9344-79889b440f5d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:30:46.652256 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91579eba-e8b8-410b-9344-79889b440f5d service cinder] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:30:46.656920 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc23eaba-743e-489c-942e-defa9bf1153c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfbb4d6809064f9cb9e06626ed7d6814', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '540a32ef88764e8c8d9201ae22f6ddaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:30:46.657548 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc23eaba-743e-489c-942e-defa9bf1153c service neutron] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:30:46.657824 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc23eaba-743e-489c-942e-defa9bf1153c service neutron] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:30:46.658092 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc23eaba-743e-489c-942e-defa9bf1153c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:30:46.660176 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-fc23eaba-743e-489c-942e-defa9bf1153c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:30:46.752967 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc23eaba-743e-489c-942e-defa9bf1153c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:30:46.755778 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc23eaba-743e-489c-942e-defa9bf1153c service neutron] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:30:46.774389 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1235/2496] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 12:30:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 206 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 12:30:49.114703 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f80d93-7f43-4023-a499-e0cf48cee186 service cinder] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:30:49.114703 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-f4f80d93-7f43-4023-a499-e0cf48cee186 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:30:49.139143 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f80d93-7f43-4023-a499-e0cf48cee186 service cinder] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:30:49.145493 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23283bd4-2706-49d7-b006-04c0633a2a3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:30:49.146933 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23283bd4-2706-49d7-b006-04c0633a2a3b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:30:49.147554 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23283bd4-2706-49d7-b006-04c0633a2a3b admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:30:49.148337 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23283bd4-2706-49d7-b006-04c0633a2a3b admin admin] PATH_INFO: `/v3/users/0020220f06d8474dad0f4d7d1d9fa397` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:30:49.150333 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23283bd4-2706-49d7-b006-04c0633a2a3b admin admin] RBAC: Authorizing `identity:delete_user(user_id=0020220f06d8474dad0f4d7d1d9fa397)` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:30:49.152678 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23283bd4-2706-49d7-b006-04c0633a2a3b admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:30:49.233814 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-23283bd4-2706-49d7-b006-04c0633a2a3b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0020220f06d8474dad0f4d7d1d9fa397'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:30:49.243434 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-23283bd4-2706-49d7-b006-04c0633a2a3b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0020220f06d8474dad0f4d7d1d9fa397'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:30:49.261242 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-23283bd4-2706-49d7-b006-04c0633a2a3b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0020220f06d8474dad0f4d7d1d9fa397'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:30:49.262015 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-23283bd4-2706-49d7-b006-04c0633a2a3b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0020220f06d8474dad0f4d7d1d9fa397'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:30:49.272470 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-23283bd4-2706-49d7-b006-04c0633a2a3b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0020220f06d8474dad0f4d7d1d9fa397'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:30:49.280588 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1262/2497] 10.222.0.15 () {58 vars in 1224 bytes} [Thu Sep 15 12:30:49 2022] DELETE /identity/v3/users/0020220f06d8474dad0f4d7d1d9fa397 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:30:49.284371 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3219174c-5b87-4d1c-a4dd-9054d682ed10 None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:30:49.284915 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3219174c-5b87-4d1c-a4dd-9054d682ed10 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:30:49.285016 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3219174c-5b87-4d1c-a4dd-9054d682ed10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:30:49.311666 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-3219174c-5b87-4d1c-a4dd-9054d682ed10 None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:30:49.314649 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-3219174c-5b87-4d1c-a4dd-9054d682ed10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:30:49.396998 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1236/2498] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 12:30:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:30:49.405831 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23283bd4-2706-49d7-b006-04c0633a2a3b admin admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:30:49.407646 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-23283bd4-2706-49d7-b006-04c0633a2a3b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:30:49.475086 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23283bd4-2706-49d7-b006-04c0633a2a3b admin admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:30:49.480249 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c944d72b-cb28-4413-96c0-8b6227d2b069 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfbb4d6809064f9cb9e06626ed7d6814', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '540a32ef88764e8c8d9201ae22f6ddaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:30:49.481487 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c944d72b-cb28-4413-96c0-8b6227d2b069 service neutron] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:30:49.481913 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c944d72b-cb28-4413-96c0-8b6227d2b069 service neutron] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:30:49.482187 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c944d72b-cb28-4413-96c0-8b6227d2b069 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:30:49.495966 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c944d72b-cb28-4413-96c0-8b6227d2b069 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:30:49.497526 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c944d72b-cb28-4413-96c0-8b6227d2b069 service neutron] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:30:49.514386 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1263/2499] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 12:30:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 12:30:49.976327 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3219174c-5b87-4d1c-a4dd-9054d682ed10 None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:30:49.981745 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-3219174c-5b87-4d1c-a4dd-9054d682ed10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:30:50.049901 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3219174c-5b87-4d1c-a4dd-9054d682ed10 None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:30:50.054701 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43ce034-b5cc-4105-98fb-9d06cb6abdd8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8191d3ccb874529b32f8d0026db629d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '540a32ef88764e8c8d9201ae22f6ddaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:30:50.055660 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43ce034-b5cc-4105-98fb-9d06cb6abdd8 service cinder] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:30:50.056100 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43ce034-b5cc-4105-98fb-9d06cb6abdd8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:30:50.057032 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43ce034-b5cc-4105-98fb-9d06cb6abdd8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:30:50.058953 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-b43ce034-b5cc-4105-98fb-9d06cb6abdd8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:30:50.066217 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:30:50.170215 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c944d72b-cb28-4413-96c0-8b6227d2b069 service neutron] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:30:50.171742 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-c944d72b-cb28-4413-96c0-8b6227d2b069 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:30:50.197491 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c944d72b-cb28-4413-96c0-8b6227d2b069 service neutron] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:30:50.207599 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43ce034-b5cc-4105-98fb-9d06cb6abdd8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:30:50.209037 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43ce034-b5cc-4105-98fb-9d06cb6abdd8 service cinder] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:30:50.209662 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79dec07a-3b63-46cb-8b89-893321676777 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:30:50.210456 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79dec07a-3b63-46cb-8b89-893321676777 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:30:50.212307 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79dec07a-3b63-46cb-8b89-893321676777 admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:30:50.212644 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79dec07a-3b63-46cb-8b89-893321676777 admin admin] PATH_INFO: `/v3/projects/aa1ddf08a3284f55a104ad7ffc100d5f` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:30:50.214939 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79dec07a-3b63-46cb-8b89-893321676777 admin admin] RBAC: Authorizing `identity:delete_project(project_id=aa1ddf08a3284f55a104ad7ffc100d5f)` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:30:50.220379 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79dec07a-3b63-46cb-8b89-893321676777 admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:30:50.227447 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:30:50.250901 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1237/2500] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 12:30:49 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 275 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 12:30:50.332395 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-79dec07a-3b63-46cb-8b89-893321676777 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aa1ddf08a3284f55a104ad7ffc100d5f', 'request_id': 'req-79dec07a-3b63-46cb-8b89-893321676777'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:30:50.350404 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-79dec07a-3b63-46cb-8b89-893321676777 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aa1ddf08a3284f55a104ad7ffc100d5f', 'request_id': 'req-79dec07a-3b63-46cb-8b89-893321676777'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:30:50.362058 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-79dec07a-3b63-46cb-8b89-893321676777 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aa1ddf08a3284f55a104ad7ffc100d5f', 'request_id': 'req-79dec07a-3b63-46cb-8b89-893321676777'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:30:50.388232 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-79dec07a-3b63-46cb-8b89-893321676777 admin admin] The token cache is being invalidate because project aa1ddf08a3284f55a104ad7ffc100d5f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81449) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 12:30:50.388857 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-79dec07a-3b63-46cb-8b89-893321676777 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:30:50.390574 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-79dec07a-3b63-46cb-8b89-893321676777 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:30:50.397117 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1264/2501] 10.222.0.15 () {58 vars in 1233 bytes} [Thu Sep 15 12:30:50 2022] DELETE /identity/v3/projects/aa1ddf08a3284f55a104ad7ffc100d5f => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:30:50.398262 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43ce034-b5cc-4105-98fb-9d06cb6abdd8 service cinder] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:30:50.419919 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43ce034-b5cc-4105-98fb-9d06cb6abdd8 service cinder] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:30:50.529200 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e6ea6d-4c5b-492b-8216-708a6fae9ba9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:30:50.530060 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e6ea6d-4c5b-492b-8216-708a6fae9ba9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:30:50.530487 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e6ea6d-4c5b-492b-8216-708a6fae9ba9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:30:50.530932 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e6ea6d-4c5b-492b-8216-708a6fae9ba9 admin admin] PATH_INFO: `/v3/users/32d0c2cbec444dcfad46dece109c27bf` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:30:50.532252 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e6ea6d-4c5b-492b-8216-708a6fae9ba9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=32d0c2cbec444dcfad46dece109c27bf)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:30:50.534382 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e6ea6d-4c5b-492b-8216-708a6fae9ba9 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:30:50.605367 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-e5e6ea6d-4c5b-492b-8216-708a6fae9ba9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '32d0c2cbec444dcfad46dece109c27bf'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:30:50.620825 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-e5e6ea6d-4c5b-492b-8216-708a6fae9ba9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '32d0c2cbec444dcfad46dece109c27bf'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:30:50.633878 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-e5e6ea6d-4c5b-492b-8216-708a6fae9ba9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '32d0c2cbec444dcfad46dece109c27bf'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:30:50.634739 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-e5e6ea6d-4c5b-492b-8216-708a6fae9ba9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '32d0c2cbec444dcfad46dece109c27bf'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:30:50.651901 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-e5e6ea6d-4c5b-492b-8216-708a6fae9ba9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '32d0c2cbec444dcfad46dece109c27bf'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:30:50.661439 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1238/2502] 10.222.0.15 () {58 vars in 1224 bytes} [Thu Sep 15 12:30:50 2022] DELETE /identity/v3/users/32d0c2cbec444dcfad46dece109c27bf => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 12:30:50.987018 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79dec07a-3b63-46cb-8b89-893321676777 admin admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:30:50.988797 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-79dec07a-3b63-46cb-8b89-893321676777 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:30:51.002626 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79dec07a-3b63-46cb-8b89-893321676777 admin admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:30:51.065771 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e6ea6d-4c5b-492b-8216-708a6fae9ba9 admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:30:51.067552 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-e5e6ea6d-4c5b-492b-8216-708a6fae9ba9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:30:51.075124 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96c3240-056f-40eb-8f85-6ee3b700f836 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:30:51.075838 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96c3240-056f-40eb-8f85-6ee3b700f836 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:30:51.076125 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96c3240-056f-40eb-8f85-6ee3b700f836 admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:30:51.076459 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96c3240-056f-40eb-8f85-6ee3b700f836 admin admin] PATH_INFO: `/v3/projects/9a27904949a047e08aea23e5b60dcbc3` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:30:51.077706 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96c3240-056f-40eb-8f85-6ee3b700f836 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9a27904949a047e08aea23e5b60dcbc3)` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:30:51.079579 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96c3240-056f-40eb-8f85-6ee3b700f836 admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:30:51.100347 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:30:51.127847 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-d96c3240-056f-40eb-8f85-6ee3b700f836 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9a27904949a047e08aea23e5b60dcbc3', 'request_id': 'req-d96c3240-056f-40eb-8f85-6ee3b700f836'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:30:51.135310 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-d96c3240-056f-40eb-8f85-6ee3b700f836 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9a27904949a047e08aea23e5b60dcbc3', 'request_id': 'req-d96c3240-056f-40eb-8f85-6ee3b700f836'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:30:51.140157 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e6ea6d-4c5b-492b-8216-708a6fae9ba9 admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:30:51.141321 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-d96c3240-056f-40eb-8f85-6ee3b700f836 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9a27904949a047e08aea23e5b60dcbc3', 'request_id': 'req-d96c3240-056f-40eb-8f85-6ee3b700f836'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:30:51.154080 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-d96c3240-056f-40eb-8f85-6ee3b700f836 admin admin] The token cache is being invalidate because project 9a27904949a047e08aea23e5b60dcbc3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81449) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 12:30:51.154367 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-d96c3240-056f-40eb-8f85-6ee3b700f836 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:30:51.155004 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-d96c3240-056f-40eb-8f85-6ee3b700f836 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:30:51.156516 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1265/2503] 10.222.0.15 () {58 vars in 1233 bytes} [Thu Sep 15 12:30:50 2022] DELETE /identity/v3/projects/9a27904949a047e08aea23e5b60dcbc3 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:30:51.161712 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba80ae08-66e3-4375-9be2-df9a03bf32b2 None None] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:30:51.161989 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba80ae08-66e3-4375-9be2-df9a03bf32b2 None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:30:51.162256 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba80ae08-66e3-4375-9be2-df9a03bf32b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:30:51.181059 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.auth.core [None req-ba80ae08-66e3-4375-9be2-df9a03bf32b2 None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81449) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:30:51.182248 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-ba80ae08-66e3-4375-9be2-df9a03bf32b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:30:51.205763 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:30:51.208936 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32ed167-6dce-4fb5-bb0f-41806033eaee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:30:51.209675 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32ed167-6dce-4fb5-bb0f-41806033eaee admin admin] REQUEST_METHOD: `DELETE` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:30:51.210031 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32ed167-6dce-4fb5-bb0f-41806033eaee admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:30:51.210242 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32ed167-6dce-4fb5-bb0f-41806033eaee admin admin] PATH_INFO: `/v3/users/1c42d5435a514bc0880b69fbbc163d32` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:30:51.211976 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32ed167-6dce-4fb5-bb0f-41806033eaee admin admin] RBAC: Authorizing `identity:delete_user(user_id=1c42d5435a514bc0880b69fbbc163d32)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:30:51.214184 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32ed167-6dce-4fb5-bb0f-41806033eaee admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:30:51.262019 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1266/2504] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 12:30:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 12:30:51.271433 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f60ec4-8a48-4c1e-8691-bba3bfb5631d None None] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:30:51.271715 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f60ec4-8a48-4c1e-8691-bba3bfb5631d None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:30:51.271989 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f60ec4-8a48-4c1e-8691-bba3bfb5631d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:30:51.273709 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-e32ed167-6dce-4fb5-bb0f-41806033eaee admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1c42d5435a514bc0880b69fbbc163d32'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:30:51.288900 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-e32ed167-6dce-4fb5-bb0f-41806033eaee admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1c42d5435a514bc0880b69fbbc163d32'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:30:51.294668 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.auth.core [None req-43f60ec4-8a48-4c1e-8691-bba3bfb5631d None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81449) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:30:51.295865 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-43f60ec4-8a48-4c1e-8691-bba3bfb5631d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:30:51.297419 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-e32ed167-6dce-4fb5-bb0f-41806033eaee admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1c42d5435a514bc0880b69fbbc163d32'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:30:51.298230 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-e32ed167-6dce-4fb5-bb0f-41806033eaee admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1c42d5435a514bc0880b69fbbc163d32'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:30:51.304747 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1267/2505] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 12:30:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 12:30:51.309214 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f60ec4-8a48-4c1e-8691-bba3bfb5631d None None] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:30:51.311387 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-e32ed167-6dce-4fb5-bb0f-41806033eaee admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1c42d5435a514bc0880b69fbbc163d32'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:30:51.319439 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f60ec4-8a48-4c1e-8691-bba3bfb5631d None None] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:30:51.320403 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1239/2506] 10.222.0.15 () {58 vars in 1224 bytes} [Thu Sep 15 12:30:51 2022] DELETE /identity/v3/users/1c42d5435a514bc0880b69fbbc163d32 => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 12:30:51.385404 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6937dd1c-03b9-4cbe-9d58-dced79274403 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:30:51.386062 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6937dd1c-03b9-4cbe-9d58-dced79274403 admin admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:30:51.386430 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6937dd1c-03b9-4cbe-9d58-dced79274403 admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:30:51.386704 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6937dd1c-03b9-4cbe-9d58-dced79274403 admin admin] PATH_INFO: `/v3/domains` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:30:51.387346 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6937dd1c-03b9-4cbe-9d58-dced79274403 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81449) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:30:51.387645 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6937dd1c-03b9-4cbe-9d58-dced79274403 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:30:51.389259 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6937dd1c-03b9-4cbe-9d58-dced79274403 admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:30:51.402993 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1268/2507] 10.222.0.15 () {58 vars in 1153 bytes} [Thu Sep 15 12:30:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:30:51.406799 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32ed167-6dce-4fb5-bb0f-41806033eaee admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:30:51.417857 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32ed167-6dce-4fb5-bb0f-41806033eaee admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:30:51.422842 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e325b4-e224-4ee8-a2c4-fb5a2fdfb7d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:30:51.423630 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e325b4-e224-4ee8-a2c4-fb5a2fdfb7d8 admin admin] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:30:51.423911 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e325b4-e224-4ee8-a2c4-fb5a2fdfb7d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:30:51.424179 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e325b4-e224-4ee8-a2c4-fb5a2fdfb7d8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:30:51.424943 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e325b4-e224-4ee8-a2c4-fb5a2fdfb7d8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:30:51.427058 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e325b4-e224-4ee8-a2c4-fb5a2fdfb7d8 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:30:51.469097 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1240/2508] 10.222.0.15 () {60 vars in 1153 bytes} [Thu Sep 15 12:30:51 2022] POST /identity/v3/projects => generated 367 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 12:30:51.470256 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6937dd1c-03b9-4cbe-9d58-dced79274403 admin admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:30:51.481796 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6937dd1c-03b9-4cbe-9d58-dced79274403 admin admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:30:51.531350 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e325b4-e224-4ee8-a2c4-fb5a2fdfb7d8 admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:30:51.533464 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-d7e325b4-e224-4ee8-a2c4-fb5a2fdfb7d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:30:51.591538 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:30:51.607016 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0129e2a-f30c-42eb-9f74-b2c6791308cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:30:51.608480 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0129e2a-f30c-42eb-9f74-b2c6791308cc admin admin] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:30:51.611426 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0129e2a-f30c-42eb-9f74-b2c6791308cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:30:51.611618 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0129e2a-f30c-42eb-9f74-b2c6791308cc admin admin] PATH_INFO: `/v3/users` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:30:51.612292 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0129e2a-f30c-42eb-9f74-b2c6791308cc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:30:51.617296 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0129e2a-f30c-42eb-9f74-b2c6791308cc admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:30:51.639838 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e325b4-e224-4ee8-a2c4-fb5a2fdfb7d8 admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:30:51.646258 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5279dc1c-3149-4562-889d-6ef49ee5e724 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:30:51.646694 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1269/2509] 10.222.0.15 () {60 vars in 1144 bytes} [Thu Sep 15 12:30:51 2022] POST /identity/v3/users => generated 284 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 12:30:51.646903 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5279dc1c-3149-4562-889d-6ef49ee5e724 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:30:51.647372 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5279dc1c-3149-4562-889d-6ef49ee5e724 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:30:51.647647 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5279dc1c-3149-4562-889d-6ef49ee5e724 admin admin] PATH_INFO: `/v3/projects/8d77a46e3c75435ca3632e12a4cb642e` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:30:51.648845 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5279dc1c-3149-4562-889d-6ef49ee5e724 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8d77a46e3c75435ca3632e12a4cb642e)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:30:51.650206 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0129e2a-f30c-42eb-9f74-b2c6791308cc admin admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:30:51.650865 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5279dc1c-3149-4562-889d-6ef49ee5e724 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:30:51.661948 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0129e2a-f30c-42eb-9f74-b2c6791308cc admin admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:30:51.666726 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3843490c-2f8c-42df-b0db-59b77ad5c6b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:30:51.667703 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3843490c-2f8c-42df-b0db-59b77ad5c6b0 admin admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:30:51.667984 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3843490c-2f8c-42df-b0db-59b77ad5c6b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:30:51.668535 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3843490c-2f8c-42df-b0db-59b77ad5c6b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:30:51.669209 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3843490c-2f8c-42df-b0db-59b77ad5c6b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:30:51.670783 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3843490c-2f8c-42df-b0db-59b77ad5c6b0 admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:30:51.685171 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1270/2510] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 12:30:51 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 15 12:30:51.686570 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3843490c-2f8c-42df-b0db-59b77ad5c6b0 admin admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:30:51.698241 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3843490c-2f8c-42df-b0db-59b77ad5c6b0 admin admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:30:51.710499 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-5279dc1c-3149-4562-889d-6ef49ee5e724 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8d77a46e3c75435ca3632e12a4cb642e', 'request_id': 'req-5279dc1c-3149-4562-889d-6ef49ee5e724'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:30:51.718654 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-5279dc1c-3149-4562-889d-6ef49ee5e724 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8d77a46e3c75435ca3632e12a4cb642e', 'request_id': 'req-5279dc1c-3149-4562-889d-6ef49ee5e724'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:30:51.724169 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-5279dc1c-3149-4562-889d-6ef49ee5e724 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8d77a46e3c75435ca3632e12a4cb642e', 'request_id': 'req-5279dc1c-3149-4562-889d-6ef49ee5e724'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:30:51.724463 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:30:51.736019 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-5279dc1c-3149-4562-889d-6ef49ee5e724 admin admin] The token cache is being invalidate because project 8d77a46e3c75435ca3632e12a4cb642e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81448) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 12:30:51.736307 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-5279dc1c-3149-4562-889d-6ef49ee5e724 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:30:51.736990 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-5279dc1c-3149-4562-889d-6ef49ee5e724 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:30:51.739132 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1241/2511] 10.222.0.15 () {58 vars in 1233 bytes} [Thu Sep 15 12:30:51 2022] DELETE /identity/v3/projects/8d77a46e3c75435ca3632e12a4cb642e => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:30:51.743818 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86f4338-8991-44fa-8a91-52111d172bfd None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:30:51.744122 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86f4338-8991-44fa-8a91-52111d172bfd None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:30:51.744398 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86f4338-8991-44fa-8a91-52111d172bfd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:30:51.759173 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc473b4-5487-40aa-a4c0-4023a89cb7cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:30:51.759781 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc473b4-5487-40aa-a4c0-4023a89cb7cc admin admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:30:51.760058 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc473b4-5487-40aa-a4c0-4023a89cb7cc admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:30:51.760326 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc473b4-5487-40aa-a4c0-4023a89cb7cc admin admin] PATH_INFO: `/v3/roles` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:30:51.760993 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc473b4-5487-40aa-a4c0-4023a89cb7cc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:30:51.762586 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc473b4-5487-40aa-a4c0-4023a89cb7cc admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:30:51.763996 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-d86f4338-8991-44fa-8a91-52111d172bfd None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:30:51.765221 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-d86f4338-8991-44fa-8a91-52111d172bfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:30:51.775035 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1271/2512] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 12:30:51 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 12:30:51.775185 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1242/2513] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 12:30:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 12:30:51.776335 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc473b4-5487-40aa-a4c0-4023a89cb7cc admin admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:30:51.778057 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-2fc473b4-5487-40aa-a4c0-4023a89cb7cc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:30:51.781389 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2d291d-303e-4417-b35b-f9009f79db51 None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:30:51.781703 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2d291d-303e-4417-b35b-f9009f79db51 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:30:51.781957 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2d291d-303e-4417-b35b-f9009f79db51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:30:51.790590 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc473b4-5487-40aa-a4c0-4023a89cb7cc admin admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:30:51.796497 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e64c83-17f1-4d55-bca5-6d43d131053d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:30:51.797194 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e64c83-17f1-4d55-bca5-6d43d131053d admin admin] REQUEST_METHOD: `PUT` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:30:51.797472 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e64c83-17f1-4d55-bca5-6d43d131053d admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:30:51.797743 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e64c83-17f1-4d55-bca5-6d43d131053d admin admin] PATH_INFO: `/v3/projects/0f283cedff48463699a33a4c3fd43d8c/users/e59d8254f8014e29a77db91836eb4a7b/roles/0679d6628a25438d90dc4bdc0401eda2` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:30:51.802537 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-cc2d291d-303e-4417-b35b-f9009f79db51 None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:30:51.803757 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-cc2d291d-303e-4417-b35b-f9009f79db51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:30:51.811707 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1243/2514] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 12:30:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 12:30:51.812911 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e64c83-17f1-4d55-bca5-6d43d131053d admin admin] RBAC: Authorizing `identity:create_grant(project_id=0f283cedff48463699a33a4c3fd43d8c, user_id=e59d8254f8014e29a77db91836eb4a7b, role_id=0679d6628a25438d90dc4bdc0401eda2)` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:30:51.815622 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91e64c83-17f1-4d55-bca5-6d43d131053d admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:30:51.815767 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2d291d-303e-4417-b35b-f9009f79db51 None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:30:51.825442 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2d291d-303e-4417-b35b-f9009f79db51 None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:30:51.829031 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1272/2515] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 15 12:30:51 2022] PUT /identity/v3/projects/0f283cedff48463699a33a4c3fd43d8c/users/e59d8254f8014e29a77db91836eb4a7b/roles/0679d6628a25438d90dc4bdc0401eda2 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 12:30:51.839265 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89950781-728f-4851-b443-d2c0489bfdc2 None None] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:30:51.839552 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89950781-728f-4851-b443-d2c0489bfdc2 None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:30:51.839825 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89950781-728f-4851-b443-d2c0489bfdc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:30:51.866592 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.auth.core [None req-89950781-728f-4851-b443-d2c0489bfdc2 None None] MFA Rules not processed for user `e59d8254f8014e29a77db91836eb4a7b`. Rule list: `[]` (Enabled: `True`). {{(pid=81449) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:30:51.867806 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-89950781-728f-4851-b443-d2c0489bfdc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:30:51.883060 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c30187-e99b-4b63-91d2-532b3177b1e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:30:51.883672 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c30187-e99b-4b63-91d2-532b3177b1e1 admin admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:30:51.883952 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c30187-e99b-4b63-91d2-532b3177b1e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:30:51.884223 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c30187-e99b-4b63-91d2-532b3177b1e1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:30:51.884962 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c30187-e99b-4b63-91d2-532b3177b1e1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:30:51.885268 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c30187-e99b-4b63-91d2-532b3177b1e1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:30:51.886850 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c30187-e99b-4b63-91d2-532b3177b1e1 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:30:51.898301 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1244/2516] 10.222.0.15 () {58 vars in 1153 bytes} [Thu Sep 15 12:30:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 12:30:51.902727 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c30187-e99b-4b63-91d2-532b3177b1e1 admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:30:51.912513 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c30187-e99b-4b63-91d2-532b3177b1e1 admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:30:51.917354 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc37301-eaad-418a-9449-35eaf8bd336f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:30:51.917948 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc37301-eaad-418a-9449-35eaf8bd336f admin admin] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:30:51.918225 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc37301-eaad-418a-9449-35eaf8bd336f admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:30:51.918495 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc37301-eaad-418a-9449-35eaf8bd336f admin admin] PATH_INFO: `/v3/projects` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:30:51.919109 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc37301-eaad-418a-9449-35eaf8bd336f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:30:51.921001 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc37301-eaad-418a-9449-35eaf8bd336f admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:30:51.932610 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1273/2517] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 12:30:51 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 12:30:51.936964 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89950781-728f-4851-b443-d2c0489bfdc2 None None] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:30:51.947933 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89950781-728f-4851-b443-d2c0489bfdc2 None None] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:30:51.952202 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1245/2518] 10.222.0.15 () {60 vars in 1153 bytes} [Thu Sep 15 12:30:51 2022] POST /identity/v3/projects => generated 379 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 15 12:30:51.953137 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc37301-eaad-418a-9449-35eaf8bd336f admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:30:51.978672 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:30:52.013727 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39464802-8e1e-4325-8ece-af850817b4e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:30:52.014503 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39464802-8e1e-4325-8ece-af850817b4e4 admin admin] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:30:52.014884 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39464802-8e1e-4325-8ece-af850817b4e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:30:52.015345 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39464802-8e1e-4325-8ece-af850817b4e4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:30:52.016087 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39464802-8e1e-4325-8ece-af850817b4e4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:30:52.018323 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39464802-8e1e-4325-8ece-af850817b4e4 admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:30:52.026492 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc37301-eaad-418a-9449-35eaf8bd336f admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:30:52.031199 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b77b6f-1999-4e22-bce9-138b64d8fc78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:30:52.031805 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b77b6f-1999-4e22-bce9-138b64d8fc78 admin admin] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:30:52.032085 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b77b6f-1999-4e22-bce9-138b64d8fc78 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:30:52.032351 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b77b6f-1999-4e22-bce9-138b64d8fc78 admin admin] PATH_INFO: `/v3/users` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:30:52.033001 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b77b6f-1999-4e22-bce9-138b64d8fc78 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:30:52.034874 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b77b6f-1999-4e22-bce9-138b64d8fc78 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:30:52.045722 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1274/2519] 10.222.0.15 () {60 vars in 1153 bytes} [Thu Sep 15 12:30:51 2022] POST /identity/v3/projects => generated 369 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 15 12:30:52.049955 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39464802-8e1e-4325-8ece-af850817b4e4 admin admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:30:52.056955 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1246/2520] 10.222.0.15 () {60 vars in 1144 bytes} [Thu Sep 15 12:30:51 2022] POST /identity/v3/users => generated 290 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 12:30:52.060964 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b77b6f-1999-4e22-bce9-138b64d8fc78 admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:30:52.068326 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:30:52.070704 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b77b6f-1999-4e22-bce9-138b64d8fc78 admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:30:52.113063 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39464802-8e1e-4325-8ece-af850817b4e4 admin admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:30:52.118402 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722f9172-c63a-455c-be70-21f6f4e6a7fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:30:52.119010 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722f9172-c63a-455c-be70-21f6f4e6a7fd admin admin] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:30:52.119285 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722f9172-c63a-455c-be70-21f6f4e6a7fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:30:52.119568 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722f9172-c63a-455c-be70-21f6f4e6a7fd admin admin] PATH_INFO: `/v3/users` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:30:52.120180 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722f9172-c63a-455c-be70-21f6f4e6a7fd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:30:52.122127 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722f9172-c63a-455c-be70-21f6f4e6a7fd admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:30:52.128861 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56bfd6fa-6ada-4069-8b32-9a8cf32f17c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:30:52.129522 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bfd6fa-6ada-4069-8b32-9a8cf32f17c1 admin admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:30:52.129808 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bfd6fa-6ada-4069-8b32-9a8cf32f17c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:30:52.130083 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bfd6fa-6ada-4069-8b32-9a8cf32f17c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:30:52.130725 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56bfd6fa-6ada-4069-8b32-9a8cf32f17c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:30:52.132302 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56bfd6fa-6ada-4069-8b32-9a8cf32f17c1 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:30:52.142282 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1247/2521] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 12:30:52 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 15 12:30:52.143863 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1275/2522] 10.222.0.15 () {60 vars in 1144 bytes} [Thu Sep 15 12:30:52 2022] POST /identity/v3/users => generated 291 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 12:30:52.147476 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56bfd6fa-6ada-4069-8b32-9a8cf32f17c1 admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:30:52.147605 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722f9172-c63a-455c-be70-21f6f4e6a7fd admin admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:30:52.157249 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722f9172-c63a-455c-be70-21f6f4e6a7fd admin admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:30:52.157440 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56bfd6fa-6ada-4069-8b32-9a8cf32f17c1 admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:30:52.162269 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3382da8-b7a8-42c9-9744-4f778d1af888 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:30:52.162418 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7ecd65-5337-4cdb-a494-3ed4ff5cc4b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:30:52.162851 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3382da8-b7a8-42c9-9744-4f778d1af888 admin admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:30:52.162967 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7ecd65-5337-4cdb-a494-3ed4ff5cc4b8 admin admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:30:52.163128 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3382da8-b7a8-42c9-9744-4f778d1af888 admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:30:52.163225 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7ecd65-5337-4cdb-a494-3ed4ff5cc4b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:30:52.163397 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3382da8-b7a8-42c9-9744-4f778d1af888 admin admin] PATH_INFO: `/v3/roles` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:30:52.163493 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7ecd65-5337-4cdb-a494-3ed4ff5cc4b8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:30:52.164033 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7ecd65-5337-4cdb-a494-3ed4ff5cc4b8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:30:52.164158 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3382da8-b7a8-42c9-9744-4f778d1af888 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:30:52.165638 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3382da8-b7a8-42c9-9744-4f778d1af888 admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:30:52.165747 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7ecd65-5337-4cdb-a494-3ed4ff5cc4b8 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:30:52.176703 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1276/2523] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 12:30:52 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 12:30:52.177125 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1248/2524] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 12:30:52 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 15 12:30:52.180637 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3382da8-b7a8-42c9-9744-4f778d1af888 admin admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:30:52.180922 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7ecd65-5337-4cdb-a494-3ed4ff5cc4b8 admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:30:52.190427 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7ecd65-5337-4cdb-a494-3ed4ff5cc4b8 admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:30:52.190692 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3382da8-b7a8-42c9-9744-4f778d1af888 admin admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:30:52.195528 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183a4f2d-d80e-4701-83e4-c1cb2733c383 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:30:52.195675 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a917b0e0-a0fe-4546-aee8-7d5905c98690 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:30:52.196170 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183a4f2d-d80e-4701-83e4-c1cb2733c383 admin admin] REQUEST_METHOD: `PUT` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:30:52.196282 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a917b0e0-a0fe-4546-aee8-7d5905c98690 admin admin] REQUEST_METHOD: `PUT` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:30:52.196446 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a917b0e0-a0fe-4546-aee8-7d5905c98690 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:30:52.196552 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183a4f2d-d80e-4701-83e4-c1cb2733c383 admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:30:52.196720 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183a4f2d-d80e-4701-83e4-c1cb2733c383 admin admin] PATH_INFO: `/v3/projects/71b4517cdb174db08ae16d2902b04445/users/ce1ce6aadd5148708fc38b445074e27b/roles/9c769b5c82884766b2c510b1f0f1364c` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:30:52.196845 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a917b0e0-a0fe-4546-aee8-7d5905c98690 admin admin] PATH_INFO: `/v3/projects/a8ba2235ee22421c9aaa54700026038d/users/d5f621006a7a4c9a833fcafb96166ef8/roles/0679d6628a25438d90dc4bdc0401eda2` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:30:52.212871 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a917b0e0-a0fe-4546-aee8-7d5905c98690 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a8ba2235ee22421c9aaa54700026038d, user_id=d5f621006a7a4c9a833fcafb96166ef8, role_id=0679d6628a25438d90dc4bdc0401eda2)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:30:52.213056 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183a4f2d-d80e-4701-83e4-c1cb2733c383 admin admin] RBAC: Authorizing `identity:create_grant(project_id=71b4517cdb174db08ae16d2902b04445, user_id=ce1ce6aadd5148708fc38b445074e27b, role_id=9c769b5c82884766b2c510b1f0f1364c)` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:30:52.215507 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a917b0e0-a0fe-4546-aee8-7d5905c98690 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:30:52.215747 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183a4f2d-d80e-4701-83e4-c1cb2733c383 admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:30:52.230418 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1277/2525] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 15 12:30:52 2022] PUT /identity/v3/projects/71b4517cdb174db08ae16d2902b04445/users/ce1ce6aadd5148708fc38b445074e27b/roles/9c769b5c82884766b2c510b1f0f1364c => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:30:52.230683 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1249/2526] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 15 12:30:52 2022] PUT /identity/v3/projects/a8ba2235ee22421c9aaa54700026038d/users/d5f621006a7a4c9a833fcafb96166ef8/roles/0679d6628a25438d90dc4bdc0401eda2 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 12:30:52.234069 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183a4f2d-d80e-4701-83e4-c1cb2733c383 admin admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:30:52.243658 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41242de5-f8bd-44e2-b26f-a94f390a2419 None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:30:52.244006 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41242de5-f8bd-44e2-b26f-a94f390a2419 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:30:52.244591 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41242de5-f8bd-44e2-b26f-a94f390a2419 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:30:52.304041 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183a4f2d-d80e-4701-83e4-c1cb2733c383 admin admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:30:52.366656 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c5f034-aff0-4d68-95ce-4e3456cbe092 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:30:52.367338 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c5f034-aff0-4d68-95ce-4e3456cbe092 admin admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:30:52.367617 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c5f034-aff0-4d68-95ce-4e3456cbe092 admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:30:52.367886 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c5f034-aff0-4d68-95ce-4e3456cbe092 admin admin] PATH_INFO: `/v3/roles` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:30:52.368523 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c5f034-aff0-4d68-95ce-4e3456cbe092 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:30:52.370155 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c5f034-aff0-4d68-95ce-4e3456cbe092 admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:30:52.380673 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1278/2527] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 12:30:52 2022] GET /identity/v3/roles => generated 1420 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 12:30:52.384952 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c5f034-aff0-4d68-95ce-4e3456cbe092 admin admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:30:52.394788 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c5f034-aff0-4d68-95ce-4e3456cbe092 admin admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:30:52.395313 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-41242de5-f8bd-44e2-b26f-a94f390a2419 None None] MFA Rules not processed for user `d5f621006a7a4c9a833fcafb96166ef8`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:30:52.396521 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-41242de5-f8bd-44e2-b26f-a94f390a2419 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:30:52.399543 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbee303-6264-4a5e-bbdd-7e813a600efe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:30:52.400188 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbee303-6264-4a5e-bbdd-7e813a600efe admin admin] REQUEST_METHOD: `PUT` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:30:52.400459 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbee303-6264-4a5e-bbdd-7e813a600efe admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:30:52.400726 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbee303-6264-4a5e-bbdd-7e813a600efe admin admin] PATH_INFO: `/v3/projects/71b4517cdb174db08ae16d2902b04445/users/ce1ce6aadd5148708fc38b445074e27b/roles/9c769b5c82884766b2c510b1f0f1364c` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:30:52.402660 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fbee303-6264-4a5e-bbdd-7e813a600efe admin admin] RBAC: Authorizing `identity:create_grant(project_id=71b4517cdb174db08ae16d2902b04445, user_id=ce1ce6aadd5148708fc38b445074e27b, role_id=9c769b5c82884766b2c510b1f0f1364c)` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:30:52.405424 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fbee303-6264-4a5e-bbdd-7e813a600efe admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:30:52.412051 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:30:52.417689 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1279/2528] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 15 12:30:52 2022] PUT /identity/v3/projects/71b4517cdb174db08ae16d2902b04445/users/ce1ce6aadd5148708fc38b445074e27b/roles/9c769b5c82884766b2c510b1f0f1364c => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 12:30:52.427838 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b30b32-5d45-46bc-919d-471fb0b41214 None None] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:30:52.428114 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b30b32-5d45-46bc-919d-471fb0b41214 None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:30:52.428382 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b30b32-5d45-46bc-919d-471fb0b41214 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:30:52.455038 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.auth.core [None req-31b30b32-5d45-46bc-919d-471fb0b41214 None None] MFA Rules not processed for user `ce1ce6aadd5148708fc38b445074e27b`. Rule list: `[]` (Enabled: `True`). {{(pid=81449) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:30:52.456264 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-31b30b32-5d45-46bc-919d-471fb0b41214 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:30:52.457573 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:30:52.463646 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1250/2529] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 12:30:52 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 12:30:52.468192 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41242de5-f8bd-44e2-b26f-a94f390a2419 None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:30:52.479020 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41242de5-f8bd-44e2-b26f-a94f390a2419 None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:30:52.531967 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1280/2530] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 12:30:52 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:30:52.534660 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc0468a-bfe3-46c0-98fb-732da33c839d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:30:52.535911 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc0468a-bfe3-46c0-98fb-732da33c839d admin admin] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:30:52.536192 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc0468a-bfe3-46c0-98fb-732da33c839d admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:30:52.536457 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc0468a-bfe3-46c0-98fb-732da33c839d admin admin] PATH_INFO: `/v3/projects` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:30:52.537117 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc0468a-bfe3-46c0-98fb-732da33c839d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:30:52.539083 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc0468a-bfe3-46c0-98fb-732da33c839d admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:30:52.539596 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b30b32-5d45-46bc-919d-471fb0b41214 None None] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:30:52.540994 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-31b30b32-5d45-46bc-919d-471fb0b41214 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:30:52.567643 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1251/2531] 10.222.0.15 () {60 vars in 1153 bytes} [Thu Sep 15 12:30:52 2022] POST /identity/v3/projects => generated 379 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 15 12:30:52.571710 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc0468a-bfe3-46c0-98fb-732da33c839d admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:30:52.603195 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b30b32-5d45-46bc-919d-471fb0b41214 None None] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:30:52.635670 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc0468a-bfe3-46c0-98fb-732da33c839d admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:30:52.640182 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89ddb75-8c7b-44a7-b3b5-f005ccf6a2d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:30:52.640817 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89ddb75-8c7b-44a7-b3b5-f005ccf6a2d8 admin admin] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:30:52.641092 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89ddb75-8c7b-44a7-b3b5-f005ccf6a2d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:30:52.641413 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89ddb75-8c7b-44a7-b3b5-f005ccf6a2d8 admin admin] PATH_INFO: `/v3/users` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:30:52.642027 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89ddb75-8c7b-44a7-b3b5-f005ccf6a2d8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:30:52.643920 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89ddb75-8c7b-44a7-b3b5-f005ccf6a2d8 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:30:52.651426 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276cc800-9058-4a76-a515-76697c17c544 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8191d3ccb874529b32f8d0026db629d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '540a32ef88764e8c8d9201ae22f6ddaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:30:52.652058 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276cc800-9058-4a76-a515-76697c17c544 service cinder] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:30:52.652361 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276cc800-9058-4a76-a515-76697c17c544 service cinder] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:30:52.652610 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276cc800-9058-4a76-a515-76697c17c544 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:30:52.666808 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1252/2532] 10.222.0.15 () {60 vars in 1144 bytes} [Thu Sep 15 12:30:52 2022] POST /identity/v3/users => generated 296 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 12:30:52.670729 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89ddb75-8c7b-44a7-b3b5-f005ccf6a2d8 admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:30:52.680956 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89ddb75-8c7b-44a7-b3b5-f005ccf6a2d8 admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:30:52.686228 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d52d95b-7739-41cb-bef4-871c425e8e16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:30:52.686851 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d52d95b-7739-41cb-bef4-871c425e8e16 admin admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:30:52.687130 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d52d95b-7739-41cb-bef4-871c425e8e16 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:30:52.687401 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d52d95b-7739-41cb-bef4-871c425e8e16 admin admin] PATH_INFO: `/v3/roles` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:30:52.688038 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d52d95b-7739-41cb-bef4-871c425e8e16 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:30:52.689665 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d52d95b-7739-41cb-bef4-871c425e8e16 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:30:52.703217 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1253/2533] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 12:30:52 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 15 12:30:52.707159 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d52d95b-7739-41cb-bef4-871c425e8e16 admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:30:52.715720 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:30:52.719545 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d52d95b-7739-41cb-bef4-871c425e8e16 admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:30:52.725332 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16bef2e2-a14b-4d52-9f1e-7fe49f916fda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:30:52.725974 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bef2e2-a14b-4d52-9f1e-7fe49f916fda admin admin] REQUEST_METHOD: `PUT` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:30:52.726251 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bef2e2-a14b-4d52-9f1e-7fe49f916fda admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:30:52.726518 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bef2e2-a14b-4d52-9f1e-7fe49f916fda admin admin] PATH_INFO: `/v3/projects/cca25c1a554b4f93bed4800e7037efb1/users/538910d4b61e48f885827f7f56d1c23b/roles/9c769b5c82884766b2c510b1f0f1364c` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:30:52.740965 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276cc800-9058-4a76-a515-76697c17c544 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:30:52.740965 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276cc800-9058-4a76-a515-76697c17c544 service cinder] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:30:52.743465 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1281/2534] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 12:30:52 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 204 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 15 12:30:52.743590 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16bef2e2-a14b-4d52-9f1e-7fe49f916fda admin admin] RBAC: Authorizing `identity:create_grant(project_id=cca25c1a554b4f93bed4800e7037efb1, user_id=538910d4b61e48f885827f7f56d1c23b, role_id=9c769b5c82884766b2c510b1f0f1364c)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:30:52.745605 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16bef2e2-a14b-4d52-9f1e-7fe49f916fda admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:30:52.759059 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1254/2535] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 15 12:30:52 2022] PUT /identity/v3/projects/cca25c1a554b4f93bed4800e7037efb1/users/538910d4b61e48f885827f7f56d1c23b/roles/9c769b5c82884766b2c510b1f0f1364c => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 12:30:52.762948 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276cc800-9058-4a76-a515-76697c17c544 service cinder] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:30:52.774445 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276cc800-9058-4a76-a515-76697c17c544 service cinder] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:30:52.790622 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16bef2e2-a14b-4d52-9f1e-7fe49f916fda admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:30:52.792370 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-16bef2e2-a14b-4d52-9f1e-7fe49f916fda admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:30:52.832026 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e82464-46f3-4f22-89e2-657f40be9ecb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:30:52.832642 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e82464-46f3-4f22-89e2-657f40be9ecb admin admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:30:52.832948 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e82464-46f3-4f22-89e2-657f40be9ecb admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:30:52.833224 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e82464-46f3-4f22-89e2-657f40be9ecb admin admin] PATH_INFO: `/v3/roles` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:30:52.833860 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e82464-46f3-4f22-89e2-657f40be9ecb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:30:52.834595 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:30:52.835631 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e82464-46f3-4f22-89e2-657f40be9ecb admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:30:52.849704 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1282/2536] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 12:30:52 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 12:30:52.850760 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e82464-46f3-4f22-89e2-657f40be9ecb admin admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:30:52.851622 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16bef2e2-a14b-4d52-9f1e-7fe49f916fda admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:30:52.855209 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd1d090-5759-44d7-b30e-5487a454da36 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8191d3ccb874529b32f8d0026db629d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '540a32ef88764e8c8d9201ae22f6ddaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:30:52.855924 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd1d090-5759-44d7-b30e-5487a454da36 service cinder] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:30:52.856198 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd1d090-5759-44d7-b30e-5487a454da36 service cinder] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:30:52.856465 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd1d090-5759-44d7-b30e-5487a454da36 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:30:52.860506 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e82464-46f3-4f22-89e2-657f40be9ecb admin admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:30:52.865747 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2fb647-e9cc-4436-b02c-48081f73546e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:30:52.866387 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2fb647-e9cc-4436-b02c-48081f73546e admin admin] REQUEST_METHOD: `PUT` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:30:52.866666 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2fb647-e9cc-4436-b02c-48081f73546e admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:30:52.866940 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2fb647-e9cc-4436-b02c-48081f73546e admin admin] PATH_INFO: `/v3/projects/cca25c1a554b4f93bed4800e7037efb1/users/538910d4b61e48f885827f7f56d1c23b/roles/9c769b5c82884766b2c510b1f0f1364c` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:30:52.869009 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea2fb647-e9cc-4436-b02c-48081f73546e admin admin] RBAC: Authorizing `identity:create_grant(project_id=cca25c1a554b4f93bed4800e7037efb1, user_id=538910d4b61e48f885827f7f56d1c23b, role_id=9c769b5c82884766b2c510b1f0f1364c)` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:30:52.871611 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea2fb647-e9cc-4436-b02c-48081f73546e admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:30:52.883010 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1283/2537] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 15 12:30:52 2022] PUT /identity/v3/projects/cca25c1a554b4f93bed4800e7037efb1/users/538910d4b61e48f885827f7f56d1c23b/roles/9c769b5c82884766b2c510b1f0f1364c => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:30:52.894295 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752d9734-5e3d-41f3-a3aa-467921bed3ee None None] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:30:52.894577 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752d9734-5e3d-41f3-a3aa-467921bed3ee None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:30:52.894849 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752d9734-5e3d-41f3-a3aa-467921bed3ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:30:52.916406 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd1d090-5759-44d7-b30e-5487a454da36 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:30:52.917848 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd1d090-5759-44d7-b30e-5487a454da36 service cinder] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:30:52.922642 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.auth.core [None req-752d9734-5e3d-41f3-a3aa-467921bed3ee None None] MFA Rules not processed for user `538910d4b61e48f885827f7f56d1c23b`. Rule list: `[]` (Enabled: `True`). {{(pid=81449) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:30:52.923833 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-752d9734-5e3d-41f3-a3aa-467921bed3ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:30:52.927846 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1255/2538] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 12:30:52 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 15 12:30:53.000171 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1284/2539] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 12:30:52 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 12:30:53.011570 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd1d090-5759-44d7-b30e-5487a454da36 service cinder] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:30:53.021106 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd1d090-5759-44d7-b30e-5487a454da36 service cinder] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:30:53.075669 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-affd492d-91ec-4e16-95df-9a1894f5876e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8191d3ccb874529b32f8d0026db629d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '540a32ef88764e8c8d9201ae22f6ddaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:30:53.076282 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affd492d-91ec-4e16-95df-9a1894f5876e service cinder] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:30:53.076630 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affd492d-91ec-4e16-95df-9a1894f5876e service cinder] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:30:53.076854 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affd492d-91ec-4e16-95df-9a1894f5876e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:30:53.086511 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-affd492d-91ec-4e16-95df-9a1894f5876e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:30:53.087918 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-affd492d-91ec-4e16-95df-9a1894f5876e service cinder] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:30:53.097322 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1256/2540] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 12:30:53 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 12:30:53.174384 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752d9734-5e3d-41f3-a3aa-467921bed3ee None None] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:30:53.185508 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752d9734-5e3d-41f3-a3aa-467921bed3ee None None] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:30:53.189665 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a603ebf2-eacf-442b-96fd-d6bc3fecce60 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8191d3ccb874529b32f8d0026db629d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '540a32ef88764e8c8d9201ae22f6ddaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:30:53.190412 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a603ebf2-eacf-442b-96fd-d6bc3fecce60 service cinder] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:30:53.190814 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a603ebf2-eacf-442b-96fd-d6bc3fecce60 service cinder] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:30:53.191524 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a603ebf2-eacf-442b-96fd-d6bc3fecce60 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:30:53.261865 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a603ebf2-eacf-442b-96fd-d6bc3fecce60 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:30:53.263495 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a603ebf2-eacf-442b-96fd-d6bc3fecce60 service cinder] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:30:53.267577 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:30:53.275941 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1285/2541] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 12:30:53 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 12:30:53.855764 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c535a84b-b1dd-48e0-93eb-57e9b8681d1b None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:30:53.856242 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c535a84b-b1dd-48e0-93eb-57e9b8681d1b None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:30:53.856544 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c535a84b-b1dd-48e0-93eb-57e9b8681d1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:30:53.880046 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-c535a84b-b1dd-48e0-93eb-57e9b8681d1b None None] MFA Rules not processed for user `527f4dd6031546c785bc6360dacb60af`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:30:53.881300 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-c535a84b-b1dd-48e0-93eb-57e9b8681d1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:30:53.944374 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1257/2542] 10.222.0.15 () {58 vars in 1009 bytes} [Thu Sep 15 12:30:53 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:30:53.998936 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a603ebf2-eacf-442b-96fd-d6bc3fecce60 service cinder] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:30:54.000214 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-a603ebf2-eacf-442b-96fd-d6bc3fecce60 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:30:54.076524 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a603ebf2-eacf-442b-96fd-d6bc3fecce60 service cinder] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:30:54.081527 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcec8dd2-31be-4331-84ba-dda7ec47a689 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa0d3dc5b7540d59fdd59bbf64a04b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '540a32ef88764e8c8d9201ae22f6ddaf', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:30:54.082134 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcec8dd2-31be-4331-84ba-dda7ec47a689 service swift] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:30:54.082411 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcec8dd2-31be-4331-84ba-dda7ec47a689 service swift] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:30:54.082680 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcec8dd2-31be-4331-84ba-dda7ec47a689 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:30:54.092025 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcec8dd2-31be-4331-84ba-dda7ec47a689 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:30:54.093888 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcec8dd2-31be-4331-84ba-dda7ec47a689 service swift] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:30:54.102950 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1286/2543] 10.222.0.15 () {60 vars in 1384 bytes} [Thu Sep 15 12:30:53 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 15 12:30:54.254565 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c535a84b-b1dd-48e0-93eb-57e9b8681d1b None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:30:54.256003 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-c535a84b-b1dd-48e0-93eb-57e9b8681d1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:30:54.317101 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:30:54.362204 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c535a84b-b1dd-48e0-93eb-57e9b8681d1b None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:30:54.368844 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42644d37-8433-4920-a30f-a3d234b5fdc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:30:54.369767 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42644d37-8433-4920-a30f-a3d234b5fdc9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:30:54.370126 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42644d37-8433-4920-a30f-a3d234b5fdc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:30:54.370885 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42644d37-8433-4920-a30f-a3d234b5fdc9 admin admin] PATH_INFO: `/v3/users/c50bb926f5fb44abad5bd52183b9538b` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:30:54.372368 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42644d37-8433-4920-a30f-a3d234b5fdc9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c50bb926f5fb44abad5bd52183b9538b)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:30:54.375199 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42644d37-8433-4920-a30f-a3d234b5fdc9 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:30:54.450280 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-42644d37-8433-4920-a30f-a3d234b5fdc9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c50bb926f5fb44abad5bd52183b9538b'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:30:54.465132 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-42644d37-8433-4920-a30f-a3d234b5fdc9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c50bb926f5fb44abad5bd52183b9538b'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:30:54.479032 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-42644d37-8433-4920-a30f-a3d234b5fdc9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c50bb926f5fb44abad5bd52183b9538b'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:30:54.479923 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-42644d37-8433-4920-a30f-a3d234b5fdc9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c50bb926f5fb44abad5bd52183b9538b'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:30:54.490176 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-42644d37-8433-4920-a30f-a3d234b5fdc9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c50bb926f5fb44abad5bd52183b9538b'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:30:54.498001 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1258/2544] 10.222.0.15 () {58 vars in 1224 bytes} [Thu Sep 15 12:30:54 2022] DELETE /identity/v3/users/c50bb926f5fb44abad5bd52183b9538b => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:30:54.502841 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821c9384-ef61-4fdb-98f9-c8fb0cfc212c None None] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:30:54.503297 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821c9384-ef61-4fdb-98f9-c8fb0cfc212c None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:30:54.503631 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821c9384-ef61-4fdb-98f9-c8fb0cfc212c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:30:54.538287 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.auth.core [None req-821c9384-ef61-4fdb-98f9-c8fb0cfc212c None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81449) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:30:54.541209 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-821c9384-ef61-4fdb-98f9-c8fb0cfc212c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:30:54.658732 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1287/2545] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 12:30:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:30:54.668261 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42644d37-8433-4920-a30f-a3d234b5fdc9 admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:30:54.671083 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-42644d37-8433-4920-a30f-a3d234b5fdc9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:30:54.795875 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42644d37-8433-4920-a30f-a3d234b5fdc9 admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:30:54.802073 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453c9a65-bb6b-4963-a1ee-5268f40f276f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfbb4d6809064f9cb9e06626ed7d6814', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '540a32ef88764e8c8d9201ae22f6ddaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:30:54.803477 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453c9a65-bb6b-4963-a1ee-5268f40f276f service neutron] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:30:54.804075 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453c9a65-bb6b-4963-a1ee-5268f40f276f service neutron] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:30:54.804582 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453c9a65-bb6b-4963-a1ee-5268f40f276f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:30:54.816333 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453c9a65-bb6b-4963-a1ee-5268f40f276f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:30:54.818484 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453c9a65-bb6b-4963-a1ee-5268f40f276f service neutron] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:30:54.833419 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1259/2546] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 12:30:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 12:30:55.273689 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821c9384-ef61-4fdb-98f9-c8fb0cfc212c None None] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:30:55.277893 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-821c9384-ef61-4fdb-98f9-c8fb0cfc212c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:30:55.309493 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821c9384-ef61-4fdb-98f9-c8fb0cfc212c None None] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:30:55.317439 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df4ee47-fad2-4e09-86a0-191dca32b0d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:30:55.318403 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df4ee47-fad2-4e09-86a0-191dca32b0d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:30:55.326716 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df4ee47-fad2-4e09-86a0-191dca32b0d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:30:55.327261 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df4ee47-fad2-4e09-86a0-191dca32b0d4 admin admin] PATH_INFO: `/v3/projects/e6b1ebf72aa14eda9e492c6375626f97` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:30:55.328457 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df4ee47-fad2-4e09-86a0-191dca32b0d4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e6b1ebf72aa14eda9e492c6375626f97)` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:30:55.330563 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df4ee47-fad2-4e09-86a0-191dca32b0d4 admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:30:55.389036 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-8df4ee47-fad2-4e09-86a0-191dca32b0d4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e6b1ebf72aa14eda9e492c6375626f97', 'request_id': 'req-8df4ee47-fad2-4e09-86a0-191dca32b0d4'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:30:55.401789 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-8df4ee47-fad2-4e09-86a0-191dca32b0d4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e6b1ebf72aa14eda9e492c6375626f97', 'request_id': 'req-8df4ee47-fad2-4e09-86a0-191dca32b0d4'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:30:55.408701 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-8df4ee47-fad2-4e09-86a0-191dca32b0d4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e6b1ebf72aa14eda9e492c6375626f97', 'request_id': 'req-8df4ee47-fad2-4e09-86a0-191dca32b0d4'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:30:55.422134 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-8df4ee47-fad2-4e09-86a0-191dca32b0d4 admin admin] The token cache is being invalidate because project e6b1ebf72aa14eda9e492c6375626f97 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81449) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 12:30:55.422708 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-8df4ee47-fad2-4e09-86a0-191dca32b0d4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:30:55.423563 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-8df4ee47-fad2-4e09-86a0-191dca32b0d4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:30:55.426619 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1288/2547] 10.222.0.15 () {58 vars in 1233 bytes} [Thu Sep 15 12:30:55 2022] DELETE /identity/v3/projects/e6b1ebf72aa14eda9e492c6375626f97 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:30:55.430308 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453c9a65-bb6b-4963-a1ee-5268f40f276f service neutron] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:30:55.442141 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453c9a65-bb6b-4963-a1ee-5268f40f276f service neutron] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:30:55.506993 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d25332-11bc-4a0a-9a52-31305796625e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:30:55.507853 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d25332-11bc-4a0a-9a52-31305796625e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:30:55.508212 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d25332-11bc-4a0a-9a52-31305796625e admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:30:55.508545 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d25332-11bc-4a0a-9a52-31305796625e admin admin] PATH_INFO: `/v3/users/f2eaab9e71da431d9edfddad27095421` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:30:55.509759 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d25332-11bc-4a0a-9a52-31305796625e admin admin] RBAC: Authorizing `identity:delete_user(user_id=f2eaab9e71da431d9edfddad27095421)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:30:55.511756 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d25332-11bc-4a0a-9a52-31305796625e admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:30:55.514919 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:30:55.570433 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-c2d25332-11bc-4a0a-9a52-31305796625e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f2eaab9e71da431d9edfddad27095421'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:30:55.586322 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-c2d25332-11bc-4a0a-9a52-31305796625e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f2eaab9e71da431d9edfddad27095421'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:30:55.595377 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-c2d25332-11bc-4a0a-9a52-31305796625e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f2eaab9e71da431d9edfddad27095421'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:30:55.596147 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-c2d25332-11bc-4a0a-9a52-31305796625e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f2eaab9e71da431d9edfddad27095421'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:30:55.605776 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-c2d25332-11bc-4a0a-9a52-31305796625e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f2eaab9e71da431d9edfddad27095421'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:30:55.612442 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1260/2548] 10.222.0.15 () {58 vars in 1224 bytes} [Thu Sep 15 12:30:55 2022] DELETE /identity/v3/users/f2eaab9e71da431d9edfddad27095421 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:30:55.926168 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df4ee47-fad2-4e09-86a0-191dca32b0d4 admin admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:30:55.927825 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-8df4ee47-fad2-4e09-86a0-191dca32b0d4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:30:55.948487 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df4ee47-fad2-4e09-86a0-191dca32b0d4 admin admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:30:55.962911 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:30:56.041211 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357456fd-d189-4202-87fc-fe45ef56c575 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:30:56.041621 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357456fd-d189-4202-87fc-fe45ef56c575 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:30:56.042005 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357456fd-d189-4202-87fc-fe45ef56c575 admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:30:56.042405 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357456fd-d189-4202-87fc-fe45ef56c575 admin admin] PATH_INFO: `/v3/projects/a2a4a011f99c432c9144184d08c15e30` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:30:56.043673 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357456fd-d189-4202-87fc-fe45ef56c575 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a2a4a011f99c432c9144184d08c15e30)` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:30:56.045759 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357456fd-d189-4202-87fc-fe45ef56c575 admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:30:56.129807 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-357456fd-d189-4202-87fc-fe45ef56c575 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a2a4a011f99c432c9144184d08c15e30', 'request_id': 'req-357456fd-d189-4202-87fc-fe45ef56c575'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:30:56.145347 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-357456fd-d189-4202-87fc-fe45ef56c575 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a2a4a011f99c432c9144184d08c15e30', 'request_id': 'req-357456fd-d189-4202-87fc-fe45ef56c575'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:30:56.156220 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-357456fd-d189-4202-87fc-fe45ef56c575 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a2a4a011f99c432c9144184d08c15e30', 'request_id': 'req-357456fd-d189-4202-87fc-fe45ef56c575'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:30:56.177861 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-357456fd-d189-4202-87fc-fe45ef56c575 admin admin] The token cache is being invalidate because project a2a4a011f99c432c9144184d08c15e30 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81449) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 12:30:56.178226 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-357456fd-d189-4202-87fc-fe45ef56c575 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:30:56.178895 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-357456fd-d189-4202-87fc-fe45ef56c575 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:30:56.192885 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1289/2549] 10.222.0.15 () {58 vars in 1233 bytes} [Thu Sep 15 12:30:55 2022] DELETE /identity/v3/projects/a2a4a011f99c432c9144184d08c15e30 => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:30:56.197656 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d25332-11bc-4a0a-9a52-31305796625e admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:30:56.199255 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-c2d25332-11bc-4a0a-9a52-31305796625e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:30:56.225834 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d25332-11bc-4a0a-9a52-31305796625e admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:30:56.321157 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f8bac5-d688-4fe2-be69-ea6fd203f25e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:30:56.322154 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f8bac5-d688-4fe2-be69-ea6fd203f25e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:30:56.322531 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f8bac5-d688-4fe2-be69-ea6fd203f25e admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:30:56.322963 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f8bac5-d688-4fe2-be69-ea6fd203f25e admin admin] PATH_INFO: `/v3/users/2afa74bb7718438fa45e697e99c349e7` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:30:56.325979 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f8bac5-d688-4fe2-be69-ea6fd203f25e admin admin] RBAC: Authorizing `identity:delete_user(user_id=2afa74bb7718438fa45e697e99c349e7)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:30:56.328184 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f8bac5-d688-4fe2-be69-ea6fd203f25e admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:30:56.412258 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-62f8bac5-d688-4fe2-be69-ea6fd203f25e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2afa74bb7718438fa45e697e99c349e7'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:30:56.435039 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-62f8bac5-d688-4fe2-be69-ea6fd203f25e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2afa74bb7718438fa45e697e99c349e7'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:30:56.455083 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-62f8bac5-d688-4fe2-be69-ea6fd203f25e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2afa74bb7718438fa45e697e99c349e7'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:30:56.457468 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-62f8bac5-d688-4fe2-be69-ea6fd203f25e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2afa74bb7718438fa45e697e99c349e7'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:30:56.468639 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-62f8bac5-d688-4fe2-be69-ea6fd203f25e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2afa74bb7718438fa45e697e99c349e7'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:30:56.471835 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:30:56.477589 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1261/2550] 10.222.0.15 () {58 vars in 1224 bytes} [Thu Sep 15 12:30:56 2022] DELETE /identity/v3/users/2afa74bb7718438fa45e697e99c349e7 => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 12:30:56.851191 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357456fd-d189-4202-87fc-fe45ef56c575 admin admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:30:56.852695 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-357456fd-d189-4202-87fc-fe45ef56c575 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:30:56.865098 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357456fd-d189-4202-87fc-fe45ef56c575 admin admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:30:56.897273 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f8bac5-d688-4fe2-be69-ea6fd203f25e admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:30:56.899605 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-62f8bac5-d688-4fe2-be69-ea6fd203f25e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:30:56.948112 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e7c329-c161-4e0d-806a-71694ed225d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:30:56.948881 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e7c329-c161-4e0d-806a-71694ed225d0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:30:56.949228 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e7c329-c161-4e0d-806a-71694ed225d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:30:56.949566 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e7c329-c161-4e0d-806a-71694ed225d0 admin admin] PATH_INFO: `/v3/users/e59d8254f8014e29a77db91836eb4a7b` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:30:56.950762 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e7c329-c161-4e0d-806a-71694ed225d0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e59d8254f8014e29a77db91836eb4a7b)` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:30:56.952747 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e7c329-c161-4e0d-806a-71694ed225d0 admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:30:57.023087 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f8bac5-d688-4fe2-be69-ea6fd203f25e admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:30:57.027354 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:30:57.030381 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6ec006-771c-4708-b39a-35c04db4a214 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:30:57.030782 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-79e7c329-c161-4e0d-806a-71694ed225d0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e59d8254f8014e29a77db91836eb4a7b'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:30:57.031186 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6ec006-771c-4708-b39a-35c04db4a214 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:30:57.031462 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6ec006-771c-4708-b39a-35c04db4a214 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:30:57.031735 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6ec006-771c-4708-b39a-35c04db4a214 admin admin] PATH_INFO: `/v3/projects/ba700df8e55b4753a23c876d06fd042a` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:30:57.033372 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6ec006-771c-4708-b39a-35c04db4a214 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ba700df8e55b4753a23c876d06fd042a)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:30:57.035369 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6ec006-771c-4708-b39a-35c04db4a214 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:30:57.041335 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-79e7c329-c161-4e0d-806a-71694ed225d0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e59d8254f8014e29a77db91836eb4a7b'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:30:57.061079 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-79e7c329-c161-4e0d-806a-71694ed225d0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e59d8254f8014e29a77db91836eb4a7b'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:30:57.061726 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-79e7c329-c161-4e0d-806a-71694ed225d0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e59d8254f8014e29a77db91836eb4a7b'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:30:57.071683 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-79e7c329-c161-4e0d-806a-71694ed225d0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e59d8254f8014e29a77db91836eb4a7b'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:30:57.081281 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1290/2551] 10.222.0.15 () {58 vars in 1224 bytes} [Thu Sep 15 12:30:56 2022] DELETE /identity/v3/users/e59d8254f8014e29a77db91836eb4a7b => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:30:57.085934 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492622a2-974d-4aae-bc2b-3663fd08e7d4 None None] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:30:57.086266 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492622a2-974d-4aae-bc2b-3663fd08e7d4 None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:30:57.086489 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492622a2-974d-4aae-bc2b-3663fd08e7d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:30:57.102204 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-8d6ec006-771c-4708-b39a-35c04db4a214 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ba700df8e55b4753a23c876d06fd042a', 'request_id': 'req-8d6ec006-771c-4708-b39a-35c04db4a214'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:30:57.110896 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.auth.core [None req-492622a2-974d-4aae-bc2b-3663fd08e7d4 None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81449) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:30:57.112485 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-492622a2-974d-4aae-bc2b-3663fd08e7d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:30:57.117084 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-8d6ec006-771c-4708-b39a-35c04db4a214 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ba700df8e55b4753a23c876d06fd042a', 'request_id': 'req-8d6ec006-771c-4708-b39a-35c04db4a214'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:30:57.131456 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-8d6ec006-771c-4708-b39a-35c04db4a214 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ba700df8e55b4753a23c876d06fd042a', 'request_id': 'req-8d6ec006-771c-4708-b39a-35c04db4a214'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:30:57.149139 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-8d6ec006-771c-4708-b39a-35c04db4a214 admin admin] The token cache is being invalidate because project ba700df8e55b4753a23c876d06fd042a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81448) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 12:30:57.149682 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-8d6ec006-771c-4708-b39a-35c04db4a214 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:30:57.150537 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-8d6ec006-771c-4708-b39a-35c04db4a214 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:30:57.153464 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1262/2552] 10.222.0.15 () {58 vars in 1233 bytes} [Thu Sep 15 12:30:56 2022] DELETE /identity/v3/projects/ba700df8e55b4753a23c876d06fd042a => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 12:30:57.160945 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfdad33-4271-4954-8f7c-6a527e8a0609 None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:30:57.162666 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfdad33-4271-4954-8f7c-6a527e8a0609 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:30:57.163626 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfdad33-4271-4954-8f7c-6a527e8a0609 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:30:57.193155 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-5dfdad33-4271-4954-8f7c-6a527e8a0609 None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:30:57.195165 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-5dfdad33-4271-4954-8f7c-6a527e8a0609 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:30:57.204707 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1291/2553] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 12:30:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 12:30:57.215607 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1263/2554] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 12:30:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:30:57.222568 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92dbc0f-ae3f-4320-a214-cd9c37b17587 None None] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:30:57.222860 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92dbc0f-ae3f-4320-a214-cd9c37b17587 None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:30:57.223140 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92dbc0f-ae3f-4320-a214-cd9c37b17587 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:30:57.226273 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfdad33-4271-4954-8f7c-6a527e8a0609 None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:30:57.227895 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-5dfdad33-4271-4954-8f7c-6a527e8a0609 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:30:57.249196 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.auth.core [None req-d92dbc0f-ae3f-4320-a214-cd9c37b17587 None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81449) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:30:57.250551 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-d92dbc0f-ae3f-4320-a214-cd9c37b17587 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:30:57.263094 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1292/2555] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 12:30:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:30:57.267398 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92dbc0f-ae3f-4320-a214-cd9c37b17587 None None] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:30:57.280395 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92dbc0f-ae3f-4320-a214-cd9c37b17587 None None] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:30:57.286610 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c44ac0-7bd5-4638-b557-c741bdbfd08e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:30:57.288223 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c44ac0-7bd5-4638-b557-c741bdbfd08e admin admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:30:57.288223 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c44ac0-7bd5-4638-b557-c741bdbfd08e admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:30:57.288223 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c44ac0-7bd5-4638-b557-c741bdbfd08e admin admin] PATH_INFO: `/v3/domains` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:30:57.288622 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c44ac0-7bd5-4638-b557-c741bdbfd08e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81449) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:30:57.288941 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c44ac0-7bd5-4638-b557-c741bdbfd08e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:30:57.290696 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c44ac0-7bd5-4638-b557-c741bdbfd08e admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:30:57.304994 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1293/2556] 10.222.0.15 () {58 vars in 1153 bytes} [Thu Sep 15 12:30:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 12:30:57.310785 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c44ac0-7bd5-4638-b557-c741bdbfd08e admin admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:30:57.324239 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfdad33-4271-4954-8f7c-6a527e8a0609 None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:30:57.325327 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c44ac0-7bd5-4638-b557-c741bdbfd08e admin admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:30:57.330316 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9908e01-4a47-4dbf-b1ef-865aa028cf59 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfbb4d6809064f9cb9e06626ed7d6814', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '540a32ef88764e8c8d9201ae22f6ddaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:30:57.331352 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9908e01-4a47-4dbf-b1ef-865aa028cf59 service neutron] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:30:57.332101 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9908e01-4a47-4dbf-b1ef-865aa028cf59 service neutron] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:30:57.332628 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9908e01-4a47-4dbf-b1ef-865aa028cf59 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:30:57.333153 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06041b5-3c7d-4111-93b6-91a48d232daa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:30:57.334268 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06041b5-3c7d-4111-93b6-91a48d232daa admin admin] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:30:57.334872 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06041b5-3c7d-4111-93b6-91a48d232daa admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:30:57.335279 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06041b5-3c7d-4111-93b6-91a48d232daa admin admin] PATH_INFO: `/v3/projects` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:30:57.336792 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06041b5-3c7d-4111-93b6-91a48d232daa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:30:57.339354 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06041b5-3c7d-4111-93b6-91a48d232daa admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:30:57.348352 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9908e01-4a47-4dbf-b1ef-865aa028cf59 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:30:57.350181 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9908e01-4a47-4dbf-b1ef-865aa028cf59 service neutron] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:30:57.380627 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1294/2557] 10.222.0.15 () {60 vars in 1153 bytes} [Thu Sep 15 12:30:57 2022] POST /identity/v3/projects => generated 389 bytes in 70 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 12:30:57.382424 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1264/2558] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 12:30:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 12:30:57.384397 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06041b5-3c7d-4111-93b6-91a48d232daa admin admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:30:57.474877 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06041b5-3c7d-4111-93b6-91a48d232daa admin admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:30:57.484580 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4e048f-9c3c-4b05-a6d4-7ca4f75bf45f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:30:57.486217 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4e048f-9c3c-4b05-a6d4-7ca4f75bf45f admin admin] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:30:57.486839 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4e048f-9c3c-4b05-a6d4-7ca4f75bf45f admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:30:57.487409 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff4e048f-9c3c-4b05-a6d4-7ca4f75bf45f admin admin] PATH_INFO: `/v3/users` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:30:57.489124 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4e048f-9c3c-4b05-a6d4-7ca4f75bf45f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:30:57.492778 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff4e048f-9c3c-4b05-a6d4-7ca4f75bf45f admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:30:57.524511 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1295/2559] 10.222.0.15 () {60 vars in 1144 bytes} [Thu Sep 15 12:30:57 2022] POST /identity/v3/users => generated 295 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 12:30:57.528646 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9908e01-4a47-4dbf-b1ef-865aa028cf59 service neutron] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:30:57.543053 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9908e01-4a47-4dbf-b1ef-865aa028cf59 service neutron] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:30:57.569742 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dce693b-ca48-4ef3-ae50-a3a50bdceb65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:30:57.570501 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dce693b-ca48-4ef3-ae50-a3a50bdceb65 admin admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:30:57.570869 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dce693b-ca48-4ef3-ae50-a3a50bdceb65 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:30:57.571547 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dce693b-ca48-4ef3-ae50-a3a50bdceb65 admin admin] PATH_INFO: `/v3/roles` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:30:57.572016 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dce693b-ca48-4ef3-ae50-a3a50bdceb65 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:30:57.574001 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dce693b-ca48-4ef3-ae50-a3a50bdceb65 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:30:57.601909 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1265/2560] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 12:30:57 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 12:30:57.611471 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4e048f-9c3c-4b05-a6d4-7ca4f75bf45f admin admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:30:57.623178 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff4e048f-9c3c-4b05-a6d4-7ca4f75bf45f admin admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:30:57.629466 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0053ec-4d83-408b-906a-c152923e685a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:30:57.630262 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0053ec-4d83-408b-906a-c152923e685a admin admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:30:57.630653 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0053ec-4d83-408b-906a-c152923e685a admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:30:57.631494 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0053ec-4d83-408b-906a-c152923e685a admin admin] PATH_INFO: `/v3/roles` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:30:57.632314 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db0053ec-4d83-408b-906a-c152923e685a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:30:57.634075 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db0053ec-4d83-408b-906a-c152923e685a admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:30:57.636903 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:DELETE=4,keystone:SELECT=91,keystone:INSERT=5 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:30:57.659502 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dce693b-ca48-4ef3-ae50-a3a50bdceb65 admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:30:57.659939 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1296/2561] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 12:30:57 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 12:30:57.672326 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dce693b-ca48-4ef3-ae50-a3a50bdceb65 admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:30:57.688972 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491a3bd5-1bfa-47a9-ae28-2116ff273a3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:30:57.689801 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491a3bd5-1bfa-47a9-ae28-2116ff273a3c admin admin] REQUEST_METHOD: `PUT` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:30:57.690174 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491a3bd5-1bfa-47a9-ae28-2116ff273a3c admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:30:57.690586 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491a3bd5-1bfa-47a9-ae28-2116ff273a3c admin admin] PATH_INFO: `/v3/projects/0102d0ccf8ef4ea3b4baf33cb85f2d82/users/44689623bb2640e49d9c7a131e02a437/roles/0679d6628a25438d90dc4bdc0401eda2` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:30:57.712981 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:30:57.721300 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-491a3bd5-1bfa-47a9-ae28-2116ff273a3c admin admin] RBAC: Authorizing `identity:create_grant(project_id=0102d0ccf8ef4ea3b4baf33cb85f2d82, user_id=44689623bb2640e49d9c7a131e02a437, role_id=0679d6628a25438d90dc4bdc0401eda2)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:30:57.725156 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-491a3bd5-1bfa-47a9-ae28-2116ff273a3c admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:30:57.732656 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0053ec-4d83-408b-906a-c152923e685a admin admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:30:57.737111 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-db0053ec-4d83-408b-906a-c152923e685a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:30:57.747941 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1266/2562] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 15 12:30:57 2022] PUT /identity/v3/projects/0102d0ccf8ef4ea3b4baf33cb85f2d82/users/44689623bb2640e49d9c7a131e02a437/roles/0679d6628a25438d90dc4bdc0401eda2 => generated 0 bytes in 89 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 12:30:57.766435 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0053ec-4d83-408b-906a-c152923e685a admin admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:30:57.767060 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d0c25a-7a22-48e8-b06d-b154e85736e7 None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:30:57.767526 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d0c25a-7a22-48e8-b06d-b154e85736e7 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:30:57.767987 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d0c25a-7a22-48e8-b06d-b154e85736e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:30:57.820730 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-41d0c25a-7a22-48e8-b06d-b154e85736e7 None None] MFA Rules not processed for user `44689623bb2640e49d9c7a131e02a437`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:30:57.823557 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-41d0c25a-7a22-48e8-b06d-b154e85736e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:30:57.883568 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97b1d0c-0fc3-4534-a61e-d5e6cbb1561a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:30:57.884516 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97b1d0c-0fc3-4534-a61e-d5e6cbb1561a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:30:57.884516 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97b1d0c-0fc3-4534-a61e-d5e6cbb1561a admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:30:57.887937 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f97b1d0c-0fc3-4534-a61e-d5e6cbb1561a admin admin] PATH_INFO: `/v3/projects/0f283cedff48463699a33a4c3fd43d8c` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:30:57.887937 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97b1d0c-0fc3-4534-a61e-d5e6cbb1561a admin admin] RBAC: Authorizing `identity:delete_project(project_id=0f283cedff48463699a33a4c3fd43d8c)` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:30:57.887937 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f97b1d0c-0fc3-4534-a61e-d5e6cbb1561a admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:30:57.946562 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1267/2563] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 12:30:57 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:30:57.951156 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d0c25a-7a22-48e8-b06d-b154e85736e7 None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:30:57.962122 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d0c25a-7a22-48e8-b06d-b154e85736e7 None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:30:57.962765 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-f97b1d0c-0fc3-4534-a61e-d5e6cbb1561a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0f283cedff48463699a33a4c3fd43d8c', 'request_id': 'req-f97b1d0c-0fc3-4534-a61e-d5e6cbb1561a'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:30:57.973590 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-f97b1d0c-0fc3-4534-a61e-d5e6cbb1561a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0f283cedff48463699a33a4c3fd43d8c', 'request_id': 'req-f97b1d0c-0fc3-4534-a61e-d5e6cbb1561a'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:30:57.981402 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-f97b1d0c-0fc3-4534-a61e-d5e6cbb1561a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0f283cedff48463699a33a4c3fd43d8c', 'request_id': 'req-f97b1d0c-0fc3-4534-a61e-d5e6cbb1561a'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:30:57.995127 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-f97b1d0c-0fc3-4534-a61e-d5e6cbb1561a admin admin] The token cache is being invalidate because project 0f283cedff48463699a33a4c3fd43d8c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81449) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 12:30:57.995895 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-f97b1d0c-0fc3-4534-a61e-d5e6cbb1561a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:30:57.996882 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-f97b1d0c-0fc3-4534-a61e-d5e6cbb1561a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:30:58.001162 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1297/2564] 10.222.0.15 () {58 vars in 1233 bytes} [Thu Sep 15 12:30:57 2022] DELETE /identity/v3/projects/0f283cedff48463699a33a4c3fd43d8c => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:30:58.002568 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97b1d0c-0fc3-4534-a61e-d5e6cbb1561a admin admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:30:58.005391 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-f97b1d0c-0fc3-4534-a61e-d5e6cbb1561a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:30:58.034467 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9b1dcc-464b-4f4f-9237-a66cda35b5b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:30:58.036087 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9b1dcc-464b-4f4f-9237-a66cda35b5b9 admin admin] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:30:58.036567 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9b1dcc-464b-4f4f-9237-a66cda35b5b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:30:58.037084 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9b1dcc-464b-4f4f-9237-a66cda35b5b9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:30:58.037917 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d9b1dcc-464b-4f4f-9237-a66cda35b5b9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:30:58.040588 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d9b1dcc-464b-4f4f-9237-a66cda35b5b9 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:30:58.077441 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1268/2565] 10.222.0.15 () {60 vars in 1153 bytes} [Thu Sep 15 12:30:57 2022] POST /identity/v3/projects => generated 389 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 15 12:30:58.081518 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9b1dcc-464b-4f4f-9237-a66cda35b5b9 admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:30:58.085173 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-2d9b1dcc-464b-4f4f-9237-a66cda35b5b9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:30:58.088762 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f97b1d0c-0fc3-4534-a61e-d5e6cbb1561a admin admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:30:58.137446 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:30:58.171554 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9b1dcc-464b-4f4f-9237-a66cda35b5b9 admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:30:58.177007 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637e04bd-4350-4460-bdee-b8901ed2d8e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:30:58.177695 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637e04bd-4350-4460-bdee-b8901ed2d8e4 admin admin] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:30:58.177976 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637e04bd-4350-4460-bdee-b8901ed2d8e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:30:58.178243 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637e04bd-4350-4460-bdee-b8901ed2d8e4 admin admin] PATH_INFO: `/v3/users` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:30:58.178875 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637e04bd-4350-4460-bdee-b8901ed2d8e4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:30:58.181956 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637e04bd-4350-4460-bdee-b8901ed2d8e4 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:30:58.182593 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd57ae2b-81d2-4bbc-a667-6b5bd84bcf9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:30:58.183326 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd57ae2b-81d2-4bbc-a667-6b5bd84bcf9b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:30:58.183677 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd57ae2b-81d2-4bbc-a667-6b5bd84bcf9b admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:30:58.184048 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd57ae2b-81d2-4bbc-a667-6b5bd84bcf9b admin admin] PATH_INFO: `/v3/users/ce1ce6aadd5148708fc38b445074e27b` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:30:58.185465 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd57ae2b-81d2-4bbc-a667-6b5bd84bcf9b admin admin] RBAC: Authorizing `identity:delete_user(user_id=ce1ce6aadd5148708fc38b445074e27b)` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:30:58.187681 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd57ae2b-81d2-4bbc-a667-6b5bd84bcf9b admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:30:58.192256 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:30:58.212904 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1269/2566] 10.222.0.15 () {60 vars in 1144 bytes} [Thu Sep 15 12:30:58 2022] POST /identity/v3/users => generated 301 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 15 12:30:58.213951 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637e04bd-4350-4460-bdee-b8901ed2d8e4 admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:30:58.228135 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637e04bd-4350-4460-bdee-b8901ed2d8e4 admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:30:58.237003 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c1efef-9aa5-4efc-b5b2-28e75f8e9c3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:30:58.239631 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c1efef-9aa5-4efc-b5b2-28e75f8e9c3e admin admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:30:58.240171 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c1efef-9aa5-4efc-b5b2-28e75f8e9c3e admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:30:58.240449 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c1efef-9aa5-4efc-b5b2-28e75f8e9c3e admin admin] PATH_INFO: `/v3/roles` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:30:58.241212 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c1efef-9aa5-4efc-b5b2-28e75f8e9c3e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:30:58.242835 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c1efef-9aa5-4efc-b5b2-28e75f8e9c3e admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:30:58.269482 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1270/2567] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 12:30:58 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 15 12:30:58.269891 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-fd57ae2b-81d2-4bbc-a667-6b5bd84bcf9b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ce1ce6aadd5148708fc38b445074e27b'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:30:58.270555 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c1efef-9aa5-4efc-b5b2-28e75f8e9c3e admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:30:58.282854 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-fd57ae2b-81d2-4bbc-a667-6b5bd84bcf9b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ce1ce6aadd5148708fc38b445074e27b'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:30:58.285415 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c1efef-9aa5-4efc-b5b2-28e75f8e9c3e admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:30:58.291334 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06cb37a-0fa3-4678-8176-162c0224db82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:30:58.292066 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06cb37a-0fa3-4678-8176-162c0224db82 admin admin] REQUEST_METHOD: `PUT` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:30:58.292353 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06cb37a-0fa3-4678-8176-162c0224db82 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:30:58.292627 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06cb37a-0fa3-4678-8176-162c0224db82 admin admin] PATH_INFO: `/v3/projects/5093030fa2ea4a61ad43c6d806a50623/users/b58a90f63d6e4e8db63de7c019d34624/roles/9c769b5c82884766b2c510b1f0f1364c` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:30:58.305400 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-fd57ae2b-81d2-4bbc-a667-6b5bd84bcf9b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ce1ce6aadd5148708fc38b445074e27b'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:30:58.306711 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-fd57ae2b-81d2-4bbc-a667-6b5bd84bcf9b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ce1ce6aadd5148708fc38b445074e27b'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:30:58.312903 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06cb37a-0fa3-4678-8176-162c0224db82 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5093030fa2ea4a61ad43c6d806a50623, user_id=b58a90f63d6e4e8db63de7c019d34624, role_id=9c769b5c82884766b2c510b1f0f1364c)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:30:58.315617 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06cb37a-0fa3-4678-8176-162c0224db82 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:30:58.316104 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-fd57ae2b-81d2-4bbc-a667-6b5bd84bcf9b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ce1ce6aadd5148708fc38b445074e27b'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:30:58.324188 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1298/2568] 10.222.0.15 () {58 vars in 1224 bytes} [Thu Sep 15 12:30:58 2022] DELETE /identity/v3/users/ce1ce6aadd5148708fc38b445074e27b => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:30:58.341061 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1271/2569] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 15 12:30:58 2022] PUT /identity/v3/projects/5093030fa2ea4a61ad43c6d806a50623/users/b58a90f63d6e4e8db63de7c019d34624/roles/9c769b5c82884766b2c510b1f0f1364c => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:30:58.344790 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd57ae2b-81d2-4bbc-a667-6b5bd84bcf9b admin admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:30:58.346816 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-fd57ae2b-81d2-4bbc-a667-6b5bd84bcf9b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:30:58.453337 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd57ae2b-81d2-4bbc-a667-6b5bd84bcf9b admin admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:30:58.534221 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a614bee7-813f-4cba-8559-4dbfdb8162c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:30:58.535882 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a614bee7-813f-4cba-8559-4dbfdb8162c5 admin admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:30:58.536386 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a614bee7-813f-4cba-8559-4dbfdb8162c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:30:58.536926 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a614bee7-813f-4cba-8559-4dbfdb8162c5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:30:58.537791 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a614bee7-813f-4cba-8559-4dbfdb8162c5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:30:58.539619 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a614bee7-813f-4cba-8559-4dbfdb8162c5 admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:30:58.542450 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:30:58.556798 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1299/2570] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 12:30:58 2022] GET /identity/v3/roles => generated 1420 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 15 12:30:58.560791 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06cb37a-0fa3-4678-8176-162c0224db82 admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:30:58.574114 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06cb37a-0fa3-4678-8176-162c0224db82 admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:30:58.581471 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d1ca12-416c-4923-b730-1ff17a62cfe9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:30:58.583176 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d1ca12-416c-4923-b730-1ff17a62cfe9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:30:58.583961 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d1ca12-416c-4923-b730-1ff17a62cfe9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:30:58.584452 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d1ca12-416c-4923-b730-1ff17a62cfe9 admin admin] PATH_INFO: `/v3/projects/5093030fa2ea4a61ad43c6d806a50623/users/b58a90f63d6e4e8db63de7c019d34624/roles/9c769b5c82884766b2c510b1f0f1364c` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:30:58.587824 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d1ca12-416c-4923-b730-1ff17a62cfe9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5093030fa2ea4a61ad43c6d806a50623, user_id=b58a90f63d6e4e8db63de7c019d34624, role_id=9c769b5c82884766b2c510b1f0f1364c)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:30:58.590845 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d1ca12-416c-4923-b730-1ff17a62cfe9 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:30:58.616700 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9631bf9d-b5e5-4136-84fe-31e389092726 None None] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:30:58.617051 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9631bf9d-b5e5-4136-84fe-31e389092726 None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:30:58.617322 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9631bf9d-b5e5-4136-84fe-31e389092726 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:30:58.619899 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1272/2571] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 15 12:30:58 2022] PUT /identity/v3/projects/5093030fa2ea4a61ad43c6d806a50623/users/b58a90f63d6e4e8db63de7c019d34624/roles/9c769b5c82884766b2c510b1f0f1364c => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 12:30:58.667213 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.auth.core [None req-9631bf9d-b5e5-4136-84fe-31e389092726 None None] MFA Rules not processed for user `b58a90f63d6e4e8db63de7c019d34624`. Rule list: `[]` (Enabled: `True`). {{(pid=81449) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:30:58.668810 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-9631bf9d-b5e5-4136-84fe-31e389092726 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:30:58.780927 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1300/2572] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 12:30:58 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:30:58.789239 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d1ca12-416c-4923-b730-1ff17a62cfe9 admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:30:58.791069 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-56d1ca12-416c-4923-b730-1ff17a62cfe9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:30:58.794869 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9631bf9d-b5e5-4136-84fe-31e389092726 None None] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:30:58.796716 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-9631bf9d-b5e5-4136-84fe-31e389092726 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:30:58.861569 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d1ca12-416c-4923-b730-1ff17a62cfe9 admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:30:58.867600 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf6fb54-8bf5-46d4-9b1b-a9eee41f1a23 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8191d3ccb874529b32f8d0026db629d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '540a32ef88764e8c8d9201ae22f6ddaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:30:58.868229 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf6fb54-8bf5-46d4-9b1b-a9eee41f1a23 service cinder] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:30:58.868507 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf6fb54-8bf5-46d4-9b1b-a9eee41f1a23 service cinder] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:30:58.868975 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf6fb54-8bf5-46d4-9b1b-a9eee41f1a23 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:30:58.884552 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf6fb54-8bf5-46d4-9b1b-a9eee41f1a23 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:30:58.886364 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf6fb54-8bf5-46d4-9b1b-a9eee41f1a23 service cinder] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:30:58.886594 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9631bf9d-b5e5-4136-84fe-31e389092726 None None] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:30:58.891980 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a9c61b-f320-4f80-bc7e-4c4b51414086 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:30:58.893366 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a9c61b-f320-4f80-bc7e-4c4b51414086 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:30:58.893726 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a9c61b-f320-4f80-bc7e-4c4b51414086 admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:30:58.894397 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a9c61b-f320-4f80-bc7e-4c4b51414086 admin admin] PATH_INFO: `/v3/projects/71b4517cdb174db08ae16d2902b04445` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:30:58.895700 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a9c61b-f320-4f80-bc7e-4c4b51414086 admin admin] RBAC: Authorizing `identity:delete_project(project_id=71b4517cdb174db08ae16d2902b04445)` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:30:58.897698 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99a9c61b-f320-4f80-bc7e-4c4b51414086 admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:30:58.898192 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1273/2573] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 12:30:58 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 12:30:58.990742 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-99a9c61b-f320-4f80-bc7e-4c4b51414086 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '71b4517cdb174db08ae16d2902b04445', 'request_id': 'req-99a9c61b-f320-4f80-bc7e-4c4b51414086'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:30:59.005281 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-99a9c61b-f320-4f80-bc7e-4c4b51414086 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '71b4517cdb174db08ae16d2902b04445', 'request_id': 'req-99a9c61b-f320-4f80-bc7e-4c4b51414086'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:30:59.013001 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-99a9c61b-f320-4f80-bc7e-4c4b51414086 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '71b4517cdb174db08ae16d2902b04445', 'request_id': 'req-99a9c61b-f320-4f80-bc7e-4c4b51414086'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:30:59.031356 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-99a9c61b-f320-4f80-bc7e-4c4b51414086 admin admin] The token cache is being invalidate because project 71b4517cdb174db08ae16d2902b04445 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81449) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 12:30:59.031675 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-99a9c61b-f320-4f80-bc7e-4c4b51414086 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:30:59.032502 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-99a9c61b-f320-4f80-bc7e-4c4b51414086 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:30:59.034309 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1301/2574] 10.222.0.15 () {58 vars in 1233 bytes} [Thu Sep 15 12:30:58 2022] DELETE /identity/v3/projects/71b4517cdb174db08ae16d2902b04445 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 12:30:59.041172 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6a062b-01a4-4dde-b888-ea8472b9638c None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:30:59.041682 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6a062b-01a4-4dde-b888-ea8472b9638c None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:30:59.042275 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac6a062b-01a4-4dde-b888-ea8472b9638c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:30:59.066327 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-ac6a062b-01a4-4dde-b888-ea8472b9638c None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:30:59.067719 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-ac6a062b-01a4-4dde-b888-ea8472b9638c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:30:59.094827 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a9c61b-f320-4f80-bc7e-4c4b51414086 admin admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:30:59.096264 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-99a9c61b-f320-4f80-bc7e-4c4b51414086 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:30:59.111595 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:30:59.151129 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1274/2575] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 12:30:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:30:59.158554 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c042021-a732-453c-bc14-f085d6ddd04e None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:30:59.158913 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c042021-a732-453c-bc14-f085d6ddd04e None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:30:59.159265 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c042021-a732-453c-bc14-f085d6ddd04e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:30:59.183185 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-5c042021-a732-453c-bc14-f085d6ddd04e None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:30:59.184475 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-5c042021-a732-453c-bc14-f085d6ddd04e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:30:59.191592 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a9c61b-f320-4f80-bc7e-4c4b51414086 admin admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:30:59.194019 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1275/2576] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 12:30:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 12:30:59.198212 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c042021-a732-453c-bc14-f085d6ddd04e None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:30:59.200912 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e2cc94-3a7f-4c5d-9fc4-248ef4c81f45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:30:59.201916 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e2cc94-3a7f-4c5d-9fc4-248ef4c81f45 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:30:59.202349 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e2cc94-3a7f-4c5d-9fc4-248ef4c81f45 admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:30:59.202762 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e2cc94-3a7f-4c5d-9fc4-248ef4c81f45 admin admin] PATH_INFO: `/v3/users/44689623bb2640e49d9c7a131e02a437` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:30:59.204097 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e2cc94-3a7f-4c5d-9fc4-248ef4c81f45 admin admin] RBAC: Authorizing `identity:delete_user(user_id=44689623bb2640e49d9c7a131e02a437)` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:30:59.206079 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e2cc94-3a7f-4c5d-9fc4-248ef4c81f45 admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:30:59.208670 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c042021-a732-453c-bc14-f085d6ddd04e None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:30:59.213965 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a373d14-ab79-422a-b93e-56488718faa3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:30:59.214791 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a373d14-ab79-422a-b93e-56488718faa3 admin admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:30:59.215251 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a373d14-ab79-422a-b93e-56488718faa3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:30:59.215743 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a373d14-ab79-422a-b93e-56488718faa3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:30:59.216560 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a373d14-ab79-422a-b93e-56488718faa3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:30:59.217084 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a373d14-ab79-422a-b93e-56488718faa3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:30:59.218836 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a373d14-ab79-422a-b93e-56488718faa3 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:30:59.242927 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1276/2577] 10.222.0.15 () {58 vars in 1153 bytes} [Thu Sep 15 12:30:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 12:30:59.244053 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a373d14-ab79-422a-b93e-56488718faa3 admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:30:59.255354 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a373d14-ab79-422a-b93e-56488718faa3 admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:30:59.260572 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd79661-3f53-4c30-b0d0-0e1430f73dc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:30:59.261394 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd79661-3f53-4c30-b0d0-0e1430f73dc6 admin admin] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:30:59.261822 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd79661-3f53-4c30-b0d0-0e1430f73dc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:30:59.262319 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd79661-3f53-4c30-b0d0-0e1430f73dc6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:30:59.263101 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd79661-3f53-4c30-b0d0-0e1430f73dc6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:30:59.265444 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd79661-3f53-4c30-b0d0-0e1430f73dc6 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:30:59.268907 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-d9e2cc94-3a7f-4c5d-9fc4-248ef4c81f45 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '44689623bb2640e49d9c7a131e02a437'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:30:59.283397 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-d9e2cc94-3a7f-4c5d-9fc4-248ef4c81f45 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '44689623bb2640e49d9c7a131e02a437'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:30:59.300177 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1277/2578] 10.222.0.15 () {60 vars in 1153 bytes} [Thu Sep 15 12:30:59 2022] POST /identity/v3/projects => generated 391 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 12:30:59.303649 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-d9e2cc94-3a7f-4c5d-9fc4-248ef4c81f45 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '44689623bb2640e49d9c7a131e02a437'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:30:59.303938 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-d9e2cc94-3a7f-4c5d-9fc4-248ef4c81f45 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '44689623bb2640e49d9c7a131e02a437'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:30:59.304872 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd79661-3f53-4c30-b0d0-0e1430f73dc6 admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:30:59.307187 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-2dd79661-3f53-4c30-b0d0-0e1430f73dc6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:30:59.314657 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-d9e2cc94-3a7f-4c5d-9fc4-248ef4c81f45 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '44689623bb2640e49d9c7a131e02a437'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:30:59.318897 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:30:59.323414 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1302/2579] 10.222.0.15 () {58 vars in 1224 bytes} [Thu Sep 15 12:30:59 2022] DELETE /identity/v3/users/44689623bb2640e49d9c7a131e02a437 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:30:59.328808 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7414a205-f537-47b8-bf90-562a0913e0db None None] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:30:59.329102 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7414a205-f537-47b8-bf90-562a0913e0db None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:30:59.329530 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7414a205-f537-47b8-bf90-562a0913e0db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:30:59.357883 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.auth.core [None req-7414a205-f537-47b8-bf90-562a0913e0db None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81449) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:30:59.359496 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-7414a205-f537-47b8-bf90-562a0913e0db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:30:59.386778 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd79661-3f53-4c30-b0d0-0e1430f73dc6 admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:30:59.443450 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1303/2580] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 12:30:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:30:59.453264 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7414a205-f537-47b8-bf90-562a0913e0db None None] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:30:59.454816 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-7414a205-f537-47b8-bf90-562a0913e0db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:30:59.477970 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca1b0a2-c6c4-4e95-bdc8-9b0da8180d1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:30:59.478598 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca1b0a2-c6c4-4e95-bdc8-9b0da8180d1e admin admin] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:30:59.478872 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca1b0a2-c6c4-4e95-bdc8-9b0da8180d1e admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:30:59.479142 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca1b0a2-c6c4-4e95-bdc8-9b0da8180d1e admin admin] PATH_INFO: `/v3/users` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:30:59.480196 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca1b0a2-c6c4-4e95-bdc8-9b0da8180d1e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:30:59.484249 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dca1b0a2-c6c4-4e95-bdc8-9b0da8180d1e admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:30:59.503249 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:30:59.516546 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1278/2581] 10.222.0.15 () {60 vars in 1144 bytes} [Thu Sep 15 12:30:59 2022] POST /identity/v3/users => generated 296 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 12:30:59.517720 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca1b0a2-c6c4-4e95-bdc8-9b0da8180d1e admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:30:59.530825 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca1b0a2-c6c4-4e95-bdc8-9b0da8180d1e admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:30:59.536696 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3e5944-ee76-4cc9-a7f8-05f95cc6c894 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:30:59.537570 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3e5944-ee76-4cc9-a7f8-05f95cc6c894 admin admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:30:59.538020 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3e5944-ee76-4cc9-a7f8-05f95cc6c894 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:30:59.538305 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3e5944-ee76-4cc9-a7f8-05f95cc6c894 admin admin] PATH_INFO: `/v3/roles` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:30:59.538950 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3e5944-ee76-4cc9-a7f8-05f95cc6c894 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:30:59.540609 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3e5944-ee76-4cc9-a7f8-05f95cc6c894 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:30:59.543114 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7414a205-f537-47b8-bf90-562a0913e0db None None] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:30:59.547201 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbab57c-63f5-4bb8-ba3c-c3a7cd4c52ae service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfbb4d6809064f9cb9e06626ed7d6814', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '540a32ef88764e8c8d9201ae22f6ddaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:30:59.547887 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbab57c-63f5-4bb8-ba3c-c3a7cd4c52ae service neutron] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:30:59.548231 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbab57c-63f5-4bb8-ba3c-c3a7cd4c52ae service neutron] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:30:59.548598 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbab57c-63f5-4bb8-ba3c-c3a7cd4c52ae service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:30:59.551629 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1279/2582] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 12:30:59 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 12:30:59.555584 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3e5944-ee76-4cc9-a7f8-05f95cc6c894 admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:30:59.563386 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bbab57c-63f5-4bb8-ba3c-c3a7cd4c52ae service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:30:59.565320 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bbab57c-63f5-4bb8-ba3c-c3a7cd4c52ae service neutron] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:30:59.565699 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3e5944-ee76-4cc9-a7f8-05f95cc6c894 admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:30:59.571140 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01db053b-9bd5-4ad2-9e46-3d506c269ae4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:30:59.571873 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01db053b-9bd5-4ad2-9e46-3d506c269ae4 admin admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:30:59.572031 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01db053b-9bd5-4ad2-9e46-3d506c269ae4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:30:59.572458 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01db053b-9bd5-4ad2-9e46-3d506c269ae4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:30:59.573131 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01db053b-9bd5-4ad2-9e46-3d506c269ae4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:30:59.574708 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01db053b-9bd5-4ad2-9e46-3d506c269ae4 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:30:59.579533 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1304/2583] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 12:30:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 15 12:30:59.586733 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1280/2584] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 12:30:59 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 15 12:30:59.590917 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbab57c-63f5-4bb8-ba3c-c3a7cd4c52ae service neutron] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:30:59.600983 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbab57c-63f5-4bb8-ba3c-c3a7cd4c52ae service neutron] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:30:59.605894 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d93f025-8f26-4528-b893-45b1a1c80764 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:30:59.606645 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d93f025-8f26-4528-b893-45b1a1c80764 admin admin] REQUEST_METHOD: `PUT` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:30:59.606972 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d93f025-8f26-4528-b893-45b1a1c80764 admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:30:59.607244 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d93f025-8f26-4528-b893-45b1a1c80764 admin admin] PATH_INFO: `/v3/projects/72ff3a200bf143f58e265995ec418f58/users/d6213a6ab5e4490cb8fd0372a7a8b495/roles/0679d6628a25438d90dc4bdc0401eda2` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:30:59.624090 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d93f025-8f26-4528-b893-45b1a1c80764 admin admin] RBAC: Authorizing `identity:create_grant(project_id=72ff3a200bf143f58e265995ec418f58, user_id=d6213a6ab5e4490cb8fd0372a7a8b495, role_id=0679d6628a25438d90dc4bdc0401eda2)` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:30:59.626778 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d93f025-8f26-4528-b893-45b1a1c80764 admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:30:59.647836 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1305/2585] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 15 12:30:59 2022] PUT /identity/v3/projects/72ff3a200bf143f58e265995ec418f58/users/d6213a6ab5e4490cb8fd0372a7a8b495/roles/0679d6628a25438d90dc4bdc0401eda2 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 12:30:59.659560 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7a7278-d9aa-4043-bef6-94c1a534522a None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:30:59.660119 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7a7278-d9aa-4043-bef6-94c1a534522a None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:30:59.660379 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7a7278-d9aa-4043-bef6-94c1a534522a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:30:59.724565 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-4c7a7278-d9aa-4043-bef6-94c1a534522a None None] MFA Rules not processed for user `d6213a6ab5e4490cb8fd0372a7a8b495`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:30:59.725832 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-4c7a7278-d9aa-4043-bef6-94c1a534522a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:30:59.888916 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d93f025-8f26-4528-b893-45b1a1c80764 admin admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:30:59.889412 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1281/2586] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 12:30:59 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 12:31:00.018026 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d93f025-8f26-4528-b893-45b1a1c80764 admin admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:31:00.024566 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7a7278-d9aa-4043-bef6-94c1a534522a None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:31:00.026223 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-4c7a7278-d9aa-4043-bef6-94c1a534522a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:31:00.029083 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86814222-c41c-4c15-8184-6f99b6af6a8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:31:00.030017 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86814222-c41c-4c15-8184-6f99b6af6a8b admin admin] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:31:00.030484 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86814222-c41c-4c15-8184-6f99b6af6a8b admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:31:00.030917 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86814222-c41c-4c15-8184-6f99b6af6a8b admin admin] PATH_INFO: `/v3/projects` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:31:00.031989 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86814222-c41c-4c15-8184-6f99b6af6a8b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:31:00.034383 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86814222-c41c-4c15-8184-6f99b6af6a8b admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:31:00.055060 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7a7278-d9aa-4043-bef6-94c1a534522a None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:31:00.065230 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f03d005-0e09-4d16-9d91-47c038c8e117 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:31:00.067516 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f03d005-0e09-4d16-9d91-47c038c8e117 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:31:00.068114 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f03d005-0e09-4d16-9d91-47c038c8e117 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:31:00.068535 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f03d005-0e09-4d16-9d91-47c038c8e117 admin admin] PATH_INFO: `/v3/projects/0102d0ccf8ef4ea3b4baf33cb85f2d82` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:31:00.069857 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f03d005-0e09-4d16-9d91-47c038c8e117 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0102d0ccf8ef4ea3b4baf33cb85f2d82)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:31:00.072037 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f03d005-0e09-4d16-9d91-47c038c8e117 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:31:00.105555 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1306/2587] 10.222.0.15 () {60 vars in 1153 bytes} [Thu Sep 15 12:30:59 2022] POST /identity/v3/projects => generated 393 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 15 12:31:00.108034 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86814222-c41c-4c15-8184-6f99b6af6a8b admin admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:31:00.132407 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:31:00.194285 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86814222-c41c-4c15-8184-6f99b6af6a8b admin admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:31:00.235521 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-0f03d005-0e09-4d16-9d91-47c038c8e117 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0102d0ccf8ef4ea3b4baf33cb85f2d82', 'request_id': 'req-0f03d005-0e09-4d16-9d91-47c038c8e117'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:31:00.261413 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-0f03d005-0e09-4d16-9d91-47c038c8e117 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0102d0ccf8ef4ea3b4baf33cb85f2d82', 'request_id': 'req-0f03d005-0e09-4d16-9d91-47c038c8e117'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:31:00.276741 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-0f03d005-0e09-4d16-9d91-47c038c8e117 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0102d0ccf8ef4ea3b4baf33cb85f2d82', 'request_id': 'req-0f03d005-0e09-4d16-9d91-47c038c8e117'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:31:00.282854 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:31:00.301083 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-0f03d005-0e09-4d16-9d91-47c038c8e117 admin admin] The token cache is being invalidate because project 0102d0ccf8ef4ea3b4baf33cb85f2d82 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81448) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 12:31:00.302019 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-0f03d005-0e09-4d16-9d91-47c038c8e117 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:31:00.302908 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-0f03d005-0e09-4d16-9d91-47c038c8e117 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:31:00.304700 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1282/2588] 10.222.0.15 () {58 vars in 1233 bytes} [Thu Sep 15 12:31:00 2022] DELETE /identity/v3/projects/0102d0ccf8ef4ea3b4baf33cb85f2d82 => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 12:31:00.308891 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f03d005-0e09-4d16-9d91-47c038c8e117 admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:31:00.310644 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-0f03d005-0e09-4d16-9d91-47c038c8e117 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:31:00.324843 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296cc3f5-954a-44e3-a96f-05a55893bd64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:31:00.325686 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296cc3f5-954a-44e3-a96f-05a55893bd64 admin admin] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:31:00.325972 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296cc3f5-954a-44e3-a96f-05a55893bd64 admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:31:00.326346 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296cc3f5-954a-44e3-a96f-05a55893bd64 admin admin] PATH_INFO: `/v3/users` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:31:00.327263 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-296cc3f5-954a-44e3-a96f-05a55893bd64 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:31:00.329580 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-296cc3f5-954a-44e3-a96f-05a55893bd64 admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:31:00.366245 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1307/2589] 10.222.0.15 () {60 vars in 1144 bytes} [Thu Sep 15 12:31:00 2022] POST /identity/v3/users => generated 303 bytes in 260 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 15 12:31:00.371176 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296cc3f5-954a-44e3-a96f-05a55893bd64 admin admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:31:00.374677 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-296cc3f5-954a-44e3-a96f-05a55893bd64 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:31:00.394210 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296cc3f5-954a-44e3-a96f-05a55893bd64 admin admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:31:00.407947 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee6978c-9340-4294-b55e-521f9718bcfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:31:00.409321 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee6978c-9340-4294-b55e-521f9718bcfe admin admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:31:00.410306 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee6978c-9340-4294-b55e-521f9718bcfe admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:31:00.410845 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee6978c-9340-4294-b55e-521f9718bcfe admin admin] PATH_INFO: `/v3/roles` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:31:00.412198 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee6978c-9340-4294-b55e-521f9718bcfe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:31:00.414051 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f03d005-0e09-4d16-9d91-47c038c8e117 admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:31:00.415807 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee6978c-9340-4294-b55e-521f9718bcfe admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:31:00.439536 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85e307f-b565-45c4-b211-d5c216a5e8fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:31:00.440647 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85e307f-b565-45c4-b211-d5c216a5e8fb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:31:00.441392 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85e307f-b565-45c4-b211-d5c216a5e8fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:31:00.441813 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85e307f-b565-45c4-b211-d5c216a5e8fb admin admin] PATH_INFO: `/v3/users/b58a90f63d6e4e8db63de7c019d34624` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:31:00.443143 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85e307f-b565-45c4-b211-d5c216a5e8fb admin admin] RBAC: Authorizing `identity:delete_user(user_id=b58a90f63d6e4e8db63de7c019d34624)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:31:00.444067 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1308/2590] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 12:31:00 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 12:31:00.447094 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85e307f-b565-45c4-b211-d5c216a5e8fb admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:31:00.448715 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee6978c-9340-4294-b55e-521f9718bcfe admin admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:31:00.464022 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee6978c-9340-4294-b55e-521f9718bcfe admin admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:31:00.473724 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55432d3-2b19-4e96-b6ff-60fb11ac2ac7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:31:00.474685 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55432d3-2b19-4e96-b6ff-60fb11ac2ac7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:31:00.475181 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55432d3-2b19-4e96-b6ff-60fb11ac2ac7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:31:00.475918 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55432d3-2b19-4e96-b6ff-60fb11ac2ac7 admin admin] PATH_INFO: `/v3/projects/5f6af8f1c76c489388fce417e83b75b7/users/9704bf1ba2424f22a5a7699f1ce33194/roles/9c769b5c82884766b2c510b1f0f1364c` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:31:00.521234 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55432d3-2b19-4e96-b6ff-60fb11ac2ac7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5f6af8f1c76c489388fce417e83b75b7, user_id=9704bf1ba2424f22a5a7699f1ce33194, role_id=9c769b5c82884766b2c510b1f0f1364c)` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:31:00.523418 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55432d3-2b19-4e96-b6ff-60fb11ac2ac7 admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:31:00.536888 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-c85e307f-b565-45c4-b211-d5c216a5e8fb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b58a90f63d6e4e8db63de7c019d34624'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:31:00.545987 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1309/2591] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 15 12:31:00 2022] PUT /identity/v3/projects/5f6af8f1c76c489388fce417e83b75b7/users/9704bf1ba2424f22a5a7699f1ce33194/roles/9c769b5c82884766b2c510b1f0f1364c => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:31:00.551983 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55432d3-2b19-4e96-b6ff-60fb11ac2ac7 admin admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:31:00.557270 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-c85e307f-b565-45c4-b211-d5c216a5e8fb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b58a90f63d6e4e8db63de7c019d34624'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:31:00.567220 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-c85e307f-b565-45c4-b211-d5c216a5e8fb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b58a90f63d6e4e8db63de7c019d34624'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:31:00.567853 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-c85e307f-b565-45c4-b211-d5c216a5e8fb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b58a90f63d6e4e8db63de7c019d34624'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:31:00.577390 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-c85e307f-b565-45c4-b211-d5c216a5e8fb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b58a90f63d6e4e8db63de7c019d34624'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:31:00.587407 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1283/2592] 10.222.0.15 () {58 vars in 1224 bytes} [Thu Sep 15 12:31:00 2022] DELETE /identity/v3/users/b58a90f63d6e4e8db63de7c019d34624 => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:31:00.662561 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55432d3-2b19-4e96-b6ff-60fb11ac2ac7 admin admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:31:00.765632 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730d2ec9-b28d-4b3f-9b4a-c091b10cab09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:31:00.766346 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730d2ec9-b28d-4b3f-9b4a-c091b10cab09 admin admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:31:00.766779 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730d2ec9-b28d-4b3f-9b4a-c091b10cab09 admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:31:00.767058 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730d2ec9-b28d-4b3f-9b4a-c091b10cab09 admin admin] PATH_INFO: `/v3/roles` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:31:00.767710 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730d2ec9-b28d-4b3f-9b4a-c091b10cab09 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:31:00.770148 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730d2ec9-b28d-4b3f-9b4a-c091b10cab09 admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:31:00.787217 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1310/2593] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 12:31:00 2022] GET /identity/v3/roles => generated 1420 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 12:31:00.791390 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85e307f-b565-45c4-b211-d5c216a5e8fb admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:31:00.792991 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-c85e307f-b565-45c4-b211-d5c216a5e8fb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:31:00.808810 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85e307f-b565-45c4-b211-d5c216a5e8fb admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:31:00.814537 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb91dc19-5d0f-4bed-87db-d4ef3fa3ef0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:31:00.815353 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb91dc19-5d0f-4bed-87db-d4ef3fa3ef0b admin admin] REQUEST_METHOD: `PUT` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:31:00.815711 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb91dc19-5d0f-4bed-87db-d4ef3fa3ef0b admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:31:00.816017 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb91dc19-5d0f-4bed-87db-d4ef3fa3ef0b admin admin] PATH_INFO: `/v3/projects/5f6af8f1c76c489388fce417e83b75b7/users/9704bf1ba2424f22a5a7699f1ce33194/roles/9c769b5c82884766b2c510b1f0f1364c` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:31:00.818091 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb91dc19-5d0f-4bed-87db-d4ef3fa3ef0b admin admin] RBAC: Authorizing `identity:create_grant(project_id=5f6af8f1c76c489388fce417e83b75b7, user_id=9704bf1ba2424f22a5a7699f1ce33194, role_id=9c769b5c82884766b2c510b1f0f1364c)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:31:00.820975 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb91dc19-5d0f-4bed-87db-d4ef3fa3ef0b admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:31:00.845534 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1284/2594] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 15 12:31:00 2022] PUT /identity/v3/projects/5f6af8f1c76c489388fce417e83b75b7/users/9704bf1ba2424f22a5a7699f1ce33194/roles/9c769b5c82884766b2c510b1f0f1364c => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:31:00.849334 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f493d83-4eae-4f39-a65a-2a7821902c2a None None] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:31:00.849795 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f493d83-4eae-4f39-a65a-2a7821902c2a None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:31:00.850227 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f493d83-4eae-4f39-a65a-2a7821902c2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:31:00.890335 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.auth.core [None req-5f493d83-4eae-4f39-a65a-2a7821902c2a None None] MFA Rules not processed for user `9704bf1ba2424f22a5a7699f1ce33194`. Rule list: `[]` (Enabled: `True`). {{(pid=81449) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:31:00.892502 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-5f493d83-4eae-4f39-a65a-2a7821902c2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:31:00.949183 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:31:00.978564 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb91dc19-5d0f-4bed-87db-d4ef3fa3ef0b admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:31:00.980508 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-eb91dc19-5d0f-4bed-87db-d4ef3fa3ef0b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:31:00.987411 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=80,keystone:DELETE=17,keystone:INSERT=3 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:31:01.025772 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1311/2595] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 12:31:00 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:31:01.040296 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f493d83-4eae-4f39-a65a-2a7821902c2a None None] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:31:01.041746 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-5f493d83-4eae-4f39-a65a-2a7821902c2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:31:01.088655 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb91dc19-5d0f-4bed-87db-d4ef3fa3ef0b admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:31:01.093890 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111ace69-0371-4c62-a027-0e64d7600abd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:31:01.095397 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111ace69-0371-4c62-a027-0e64d7600abd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:31:01.095688 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111ace69-0371-4c62-a027-0e64d7600abd admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:31:01.096036 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111ace69-0371-4c62-a027-0e64d7600abd admin admin] PATH_INFO: `/v3/projects/5093030fa2ea4a61ad43c6d806a50623` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:31:01.097735 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111ace69-0371-4c62-a027-0e64d7600abd admin admin] RBAC: Authorizing `identity:delete_project(project_id=5093030fa2ea4a61ad43c6d806a50623)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:31:01.099737 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111ace69-0371-4c62-a027-0e64d7600abd admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:31:01.114226 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f493d83-4eae-4f39-a65a-2a7821902c2a None None] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:31:01.118079 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608fea6a-2ea6-4279-8305-41c30838b7fe service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8191d3ccb874529b32f8d0026db629d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '540a32ef88764e8c8d9201ae22f6ddaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:31:01.118695 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608fea6a-2ea6-4279-8305-41c30838b7fe service cinder] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:31:01.118972 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608fea6a-2ea6-4279-8305-41c30838b7fe service cinder] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:31:01.119246 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608fea6a-2ea6-4279-8305-41c30838b7fe service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:31:01.130217 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608fea6a-2ea6-4279-8305-41c30838b7fe service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:31:01.131677 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-608fea6a-2ea6-4279-8305-41c30838b7fe service cinder] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:31:01.143854 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1312/2596] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 12:31:01 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 15 12:31:01.168052 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-111ace69-0371-4c62-a027-0e64d7600abd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5093030fa2ea4a61ad43c6d806a50623', 'request_id': 'req-111ace69-0371-4c62-a027-0e64d7600abd'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:31:01.177633 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-111ace69-0371-4c62-a027-0e64d7600abd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5093030fa2ea4a61ad43c6d806a50623', 'request_id': 'req-111ace69-0371-4c62-a027-0e64d7600abd'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:31:01.183983 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-111ace69-0371-4c62-a027-0e64d7600abd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5093030fa2ea4a61ad43c6d806a50623', 'request_id': 'req-111ace69-0371-4c62-a027-0e64d7600abd'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:31:01.199508 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-111ace69-0371-4c62-a027-0e64d7600abd admin admin] The token cache is being invalidate because project 5093030fa2ea4a61ad43c6d806a50623 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81448) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 12:31:01.199981 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-111ace69-0371-4c62-a027-0e64d7600abd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:31:01.201146 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-111ace69-0371-4c62-a027-0e64d7600abd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:31:01.203191 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1285/2597] 10.222.0.15 () {58 vars in 1233 bytes} [Thu Sep 15 12:31:00 2022] DELETE /identity/v3/projects/5093030fa2ea4a61ad43c6d806a50623 => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:31:01.208253 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945847b1-f9c1-4012-a2b5-10f5cbdcc405 None None] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:31:01.208927 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945847b1-f9c1-4012-a2b5-10f5cbdcc405 None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:31:01.209190 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945847b1-f9c1-4012-a2b5-10f5cbdcc405 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:31:01.214391 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111ace69-0371-4c62-a027-0e64d7600abd admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:31:01.216234 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-111ace69-0371-4c62-a027-0e64d7600abd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:31:01.231323 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.auth.core [None req-945847b1-f9c1-4012-a2b5-10f5cbdcc405 None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81449) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:31:01.232517 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-945847b1-f9c1-4012-a2b5-10f5cbdcc405 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:31:01.282393 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111ace69-0371-4c62-a027-0e64d7600abd admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:31:01.290445 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64234d91-bf28-4b78-b271-fe47ae0c8fd5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8191d3ccb874529b32f8d0026db629d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '540a32ef88764e8c8d9201ae22f6ddaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:31:01.291248 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64234d91-bf28-4b78-b271-fe47ae0c8fd5 service cinder] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:31:01.291777 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64234d91-bf28-4b78-b271-fe47ae0c8fd5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:31:01.292149 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64234d91-bf28-4b78-b271-fe47ae0c8fd5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:31:01.294153 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-64234d91-bf28-4b78-b271-fe47ae0c8fd5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:31:01.309756 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1313/2598] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 12:31:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:31:01.319985 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b15acfd-f09d-46b6-8c48-97be196d2f7e None None] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:31:01.320374 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b15acfd-f09d-46b6-8c48-97be196d2f7e None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:31:01.323508 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b15acfd-f09d-46b6-8c48-97be196d2f7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:31:01.353849 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:31:01.354876 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.auth.core [None req-3b15acfd-f09d-46b6-8c48-97be196d2f7e None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81449) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:31:01.357967 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-3b15acfd-f09d-46b6-8c48-97be196d2f7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:31:01.371128 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1314/2599] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 12:31:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:31:01.375865 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64234d91-bf28-4b78-b271-fe47ae0c8fd5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:31:01.376939 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b15acfd-f09d-46b6-8c48-97be196d2f7e None None] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:31:01.377614 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64234d91-bf28-4b78-b271-fe47ae0c8fd5 service cinder] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:31:01.390672 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b15acfd-f09d-46b6-8c48-97be196d2f7e None None] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:31:01.393252 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1286/2600] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 12:31:01 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 15 12:31:01.406155 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2cd8b2f-60fe-4046-9f41-eeca53c1345c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:31:01.406891 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cd8b2f-60fe-4046-9f41-eeca53c1345c admin admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:31:01.407176 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cd8b2f-60fe-4046-9f41-eeca53c1345c admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:31:01.407445 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cd8b2f-60fe-4046-9f41-eeca53c1345c admin admin] PATH_INFO: `/v3/domains` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:31:01.408161 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2cd8b2f-60fe-4046-9f41-eeca53c1345c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81449) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:31:01.408470 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2cd8b2f-60fe-4046-9f41-eeca53c1345c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:31:01.410134 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2cd8b2f-60fe-4046-9f41-eeca53c1345c admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:31:01.426380 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1315/2601] 10.222.0.15 () {58 vars in 1153 bytes} [Thu Sep 15 12:31:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:31:01.430974 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64234d91-bf28-4b78-b271-fe47ae0c8fd5 service cinder] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:31:01.443607 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64234d91-bf28-4b78-b271-fe47ae0c8fd5 service cinder] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:31:01.449149 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9e4968-f61c-4866-8559-f4dd84fd39e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:31:01.449746 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9e4968-f61c-4866-8559-f4dd84fd39e6 admin admin] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:31:01.450095 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9e4968-f61c-4866-8559-f4dd84fd39e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:31:01.450861 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9e4968-f61c-4866-8559-f4dd84fd39e6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:31:01.451188 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9e4968-f61c-4866-8559-f4dd84fd39e6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:31:01.453438 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9e4968-f61c-4866-8559-f4dd84fd39e6 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:31:01.489866 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1287/2602] 10.222.0.15 () {60 vars in 1153 bytes} [Thu Sep 15 12:31:01 2022] POST /identity/v3/projects => generated 365 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 12:31:01.495081 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2cd8b2f-60fe-4046-9f41-eeca53c1345c admin admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:31:01.511843 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2cd8b2f-60fe-4046-9f41-eeca53c1345c admin admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:31:01.597726 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:31:01.619317 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c1c509-4907-4725-a1c2-bfbb0a11f432 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:31:01.620092 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c1c509-4907-4725-a1c2-bfbb0a11f432 admin admin] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:31:01.620469 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c1c509-4907-4725-a1c2-bfbb0a11f432 admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:31:01.620741 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c1c509-4907-4725-a1c2-bfbb0a11f432 admin admin] PATH_INFO: `/v3/users` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:31:01.621807 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c1c509-4907-4725-a1c2-bfbb0a11f432 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:31:01.624346 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c1c509-4907-4725-a1c2-bfbb0a11f432 admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:31:01.670688 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1316/2603] 10.222.0.15 () {60 vars in 1144 bytes} [Thu Sep 15 12:31:01 2022] POST /identity/v3/users => generated 283 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 15 12:31:01.675803 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9e4968-f61c-4866-8559-f4dd84fd39e6 admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:31:01.694997 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9e4968-f61c-4866-8559-f4dd84fd39e6 admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:31:01.706238 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3f9cfd-a2fb-412b-b92e-c3e4bbe760e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:31:01.709842 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3f9cfd-a2fb-412b-b92e-c3e4bbe760e9 admin admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:31:01.710215 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3f9cfd-a2fb-412b-b92e-c3e4bbe760e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:31:01.710908 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3f9cfd-a2fb-412b-b92e-c3e4bbe760e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:31:01.711704 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3f9cfd-a2fb-412b-b92e-c3e4bbe760e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:31:01.713968 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3f9cfd-a2fb-412b-b92e-c3e4bbe760e9 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:31:01.744325 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1288/2604] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 12:31:01 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 12:31:01.749103 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c1c509-4907-4725-a1c2-bfbb0a11f432 admin admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:31:01.763064 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c1c509-4907-4725-a1c2-bfbb0a11f432 admin admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:31:01.780169 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770a76e0-c015-418a-9925-cb4482114484 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:31:01.780169 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770a76e0-c015-418a-9925-cb4482114484 admin admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:31:01.780169 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770a76e0-c015-418a-9925-cb4482114484 admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:31:01.780169 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770a76e0-c015-418a-9925-cb4482114484 admin admin] PATH_INFO: `/v3/roles` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:31:01.780584 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770a76e0-c015-418a-9925-cb4482114484 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:31:01.780584 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770a76e0-c015-418a-9925-cb4482114484 admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:31:01.795903 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1317/2605] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 12:31:01 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 15 12:31:01.800006 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3f9cfd-a2fb-412b-b92e-c3e4bbe760e9 admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:31:01.814485 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3f9cfd-a2fb-412b-b92e-c3e4bbe760e9 admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:31:01.846582 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9019d4-6f9c-4e97-b619-8448ec8f5997 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:31:01.849447 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9019d4-6f9c-4e97-b619-8448ec8f5997 admin admin] REQUEST_METHOD: `PUT` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:31:01.849771 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9019d4-6f9c-4e97-b619-8448ec8f5997 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:31:01.850469 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9019d4-6f9c-4e97-b619-8448ec8f5997 admin admin] PATH_INFO: `/v3/projects/cfb6d88be2484e9eab6dac4e5d216efa/users/f1300202d61b4d908508a9a8035b7366/roles/0679d6628a25438d90dc4bdc0401eda2` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:31:01.892207 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9019d4-6f9c-4e97-b619-8448ec8f5997 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cfb6d88be2484e9eab6dac4e5d216efa, user_id=f1300202d61b4d908508a9a8035b7366, role_id=0679d6628a25438d90dc4bdc0401eda2)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:31:01.895658 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b9019d4-6f9c-4e97-b619-8448ec8f5997 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:31:01.930590 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1289/2606] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 15 12:31:01 2022] PUT /identity/v3/projects/cfb6d88be2484e9eab6dac4e5d216efa/users/f1300202d61b4d908508a9a8035b7366/roles/0679d6628a25438d90dc4bdc0401eda2 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:31:01.939004 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613774a3-6ff8-4e7b-a1b6-8619a538b161 None None] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:31:01.939299 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613774a3-6ff8-4e7b-a1b6-8619a538b161 None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:31:01.939577 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613774a3-6ff8-4e7b-a1b6-8619a538b161 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:31:01.994276 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.auth.core [None req-613774a3-6ff8-4e7b-a1b6-8619a538b161 None None] MFA Rules not processed for user `f1300202d61b4d908508a9a8035b7366`. Rule list: `[]` (Enabled: `True`). {{(pid=81449) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:31:01.999283 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-613774a3-6ff8-4e7b-a1b6-8619a538b161 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:31:02.112072 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1318/2607] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 12:31:01 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:31:02.113733 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9019d4-6f9c-4e97-b619-8448ec8f5997 admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:31:02.181171 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9019d4-6f9c-4e97-b619-8448ec8f5997 admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:31:02.187513 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9298913-2d45-4dbf-a2a0-3656aa8d986f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:31:02.188191 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9298913-2d45-4dbf-a2a0-3656aa8d986f admin admin] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:31:02.188599 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9298913-2d45-4dbf-a2a0-3656aa8d986f admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:31:02.188936 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9298913-2d45-4dbf-a2a0-3656aa8d986f admin admin] PATH_INFO: `/v3/projects` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:31:02.189654 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9298913-2d45-4dbf-a2a0-3656aa8d986f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:31:02.192448 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9298913-2d45-4dbf-a2a0-3656aa8d986f admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:31:02.228893 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1290/2608] 10.222.0.15 () {60 vars in 1153 bytes} [Thu Sep 15 12:31:02 2022] POST /identity/v3/projects => generated 367 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 12:31:02.229737 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613774a3-6ff8-4e7b-a1b6-8619a538b161 None None] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:31:02.243283 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613774a3-6ff8-4e7b-a1b6-8619a538b161 None None] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:31:02.311206 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97259f6-84ba-4662-ab72-91a983cf859b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:31:02.311841 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97259f6-84ba-4662-ab72-91a983cf859b admin admin] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:31:02.312125 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97259f6-84ba-4662-ab72-91a983cf859b admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:31:02.312397 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97259f6-84ba-4662-ab72-91a983cf859b admin admin] PATH_INFO: `/v3/users` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:31:02.313089 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e97259f6-84ba-4662-ab72-91a983cf859b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:31:02.315160 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e97259f6-84ba-4662-ab72-91a983cf859b admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:31:02.345468 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1319/2609] 10.222.0.15 () {60 vars in 1144 bytes} [Thu Sep 15 12:31:02 2022] POST /identity/v3/users => generated 290 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 15 12:31:02.349113 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9298913-2d45-4dbf-a2a0-3656aa8d986f admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:31:02.361239 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9298913-2d45-4dbf-a2a0-3656aa8d986f admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:31:02.368038 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bb05a6-464c-4840-b87b-74d5896d06ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:31:02.368584 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bb05a6-464c-4840-b87b-74d5896d06ff admin admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:31:02.368921 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bb05a6-464c-4840-b87b-74d5896d06ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:31:02.369197 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4bb05a6-464c-4840-b87b-74d5896d06ff admin admin] PATH_INFO: `/v3/roles` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:31:02.369837 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4bb05a6-464c-4840-b87b-74d5896d06ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:31:02.371578 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4bb05a6-464c-4840-b87b-74d5896d06ff admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:31:02.384658 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1291/2610] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 12:31:02 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 12:31:02.389387 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97259f6-84ba-4662-ab72-91a983cf859b admin admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:31:02.401639 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97259f6-84ba-4662-ab72-91a983cf859b admin admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:31:02.406941 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8efd11a-56cc-4e34-bcba-8e53b7bcb81b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:31:02.407619 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8efd11a-56cc-4e34-bcba-8e53b7bcb81b admin admin] REQUEST_METHOD: `PUT` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:31:02.407899 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8efd11a-56cc-4e34-bcba-8e53b7bcb81b admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:31:02.408170 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8efd11a-56cc-4e34-bcba-8e53b7bcb81b admin admin] PATH_INFO: `/v3/projects/3b2b2ab214054db39da9cc7c6b97131d/users/4f3d25cec8b64e68abe3bc0d23242af1/roles/9c769b5c82884766b2c510b1f0f1364c` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:31:02.427553 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8efd11a-56cc-4e34-bcba-8e53b7bcb81b admin admin] RBAC: Authorizing `identity:create_grant(project_id=3b2b2ab214054db39da9cc7c6b97131d, user_id=4f3d25cec8b64e68abe3bc0d23242af1, role_id=9c769b5c82884766b2c510b1f0f1364c)` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:31:02.430300 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8efd11a-56cc-4e34-bcba-8e53b7bcb81b admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:31:02.446093 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1320/2611] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 15 12:31:02 2022] PUT /identity/v3/projects/3b2b2ab214054db39da9cc7c6b97131d/users/4f3d25cec8b64e68abe3bc0d23242af1/roles/9c769b5c82884766b2c510b1f0f1364c => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 12:31:02.450097 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bb05a6-464c-4840-b87b-74d5896d06ff admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:31:02.464792 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4bb05a6-464c-4840-b87b-74d5896d06ff admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:31:02.537013 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da805086-080f-41c2-b8f7-f27931486834 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:31:02.537642 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da805086-080f-41c2-b8f7-f27931486834 admin admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:31:02.537922 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da805086-080f-41c2-b8f7-f27931486834 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:31:02.538209 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da805086-080f-41c2-b8f7-f27931486834 admin admin] PATH_INFO: `/v3/roles` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:31:02.538855 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da805086-080f-41c2-b8f7-f27931486834 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:31:02.540563 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da805086-080f-41c2-b8f7-f27931486834 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:31:02.546162 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:31:02.556790 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1292/2612] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 12:31:02 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 12:31:02.560618 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8efd11a-56cc-4e34-bcba-8e53b7bcb81b admin admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:31:02.572580 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8efd11a-56cc-4e34-bcba-8e53b7bcb81b admin admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:31:02.580411 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf60df4-578c-425c-82cc-e3ac14f441e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:31:02.581942 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf60df4-578c-425c-82cc-e3ac14f441e5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:31:02.581942 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf60df4-578c-425c-82cc-e3ac14f441e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:31:02.582159 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf60df4-578c-425c-82cc-e3ac14f441e5 admin admin] PATH_INFO: `/v3/projects/3b2b2ab214054db39da9cc7c6b97131d/users/4f3d25cec8b64e68abe3bc0d23242af1/roles/9c769b5c82884766b2c510b1f0f1364c` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:31:02.585060 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf60df4-578c-425c-82cc-e3ac14f441e5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3b2b2ab214054db39da9cc7c6b97131d, user_id=4f3d25cec8b64e68abe3bc0d23242af1, role_id=9c769b5c82884766b2c510b1f0f1364c)` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:31:02.589172 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf60df4-578c-425c-82cc-e3ac14f441e5 admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:31:02.603255 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1321/2613] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 15 12:31:02 2022] PUT /identity/v3/projects/3b2b2ab214054db39da9cc7c6b97131d/users/4f3d25cec8b64e68abe3bc0d23242af1/roles/9c769b5c82884766b2c510b1f0f1364c => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:31:02.614299 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94e34ef-0332-496a-90c4-5d92c081648c None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:31:02.614588 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94e34ef-0332-496a-90c4-5d92c081648c None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:31:02.614884 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94e34ef-0332-496a-90c4-5d92c081648c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:31:02.648688 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-f94e34ef-0332-496a-90c4-5d92c081648c None None] MFA Rules not processed for user `4f3d25cec8b64e68abe3bc0d23242af1`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:31:02.650318 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-f94e34ef-0332-496a-90c4-5d92c081648c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:31:02.739719 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1293/2614] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 12:31:02 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:31:02.752341 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf60df4-578c-425c-82cc-e3ac14f441e5 admin admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:31:02.754175 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-5bf60df4-578c-425c-82cc-e3ac14f441e5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:31:02.772742 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:31:02.828701 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf60df4-578c-425c-82cc-e3ac14f441e5 admin admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:31:02.833351 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb08577b-b0e9-46a1-8203-982eecb4eecc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8191d3ccb874529b32f8d0026db629d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '540a32ef88764e8c8d9201ae22f6ddaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:31:02.834029 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb08577b-b0e9-46a1-8203-982eecb4eecc service cinder] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:31:02.834379 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb08577b-b0e9-46a1-8203-982eecb4eecc service cinder] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:31:02.834828 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb08577b-b0e9-46a1-8203-982eecb4eecc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:31:02.845444 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb08577b-b0e9-46a1-8203-982eecb4eecc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:31:02.846875 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb08577b-b0e9-46a1-8203-982eecb4eecc service cinder] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:31:02.857938 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1322/2615] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 12:31:02 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 12:31:02.953686 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94e34ef-0332-496a-90c4-5d92c081648c None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:31:02.967440 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94e34ef-0332-496a-90c4-5d92c081648c None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:31:02.977128 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f641236-6150-45ce-918f-7d52a38cae23 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8191d3ccb874529b32f8d0026db629d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '540a32ef88764e8c8d9201ae22f6ddaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:31:02.977978 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f641236-6150-45ce-918f-7d52a38cae23 service cinder] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:31:02.979015 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f641236-6150-45ce-918f-7d52a38cae23 service cinder] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:31:02.979292 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f641236-6150-45ce-918f-7d52a38cae23 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:31:03.049814 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f641236-6150-45ce-918f-7d52a38cae23 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:31:03.051272 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f641236-6150-45ce-918f-7d52a38cae23 service cinder] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:31:03.071790 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1294/2616] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 12:31:02 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 12:31:03.653605 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb08577b-b0e9-46a1-8203-982eecb4eecc service cinder] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:31:03.656241 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-fb08577b-b0e9-46a1-8203-982eecb4eecc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:31:03.894367 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb08577b-b0e9-46a1-8203-982eecb4eecc service cinder] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:31:03.902381 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a357c7d-dbcf-41ac-b0cf-a77bd7b07651 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:31:03.903202 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a357c7d-dbcf-41ac-b0cf-a77bd7b07651 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:31:03.903479 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a357c7d-dbcf-41ac-b0cf-a77bd7b07651 admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:31:03.903855 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a357c7d-dbcf-41ac-b0cf-a77bd7b07651 admin admin] PATH_INFO: `/v3/users/d5f621006a7a4c9a833fcafb96166ef8` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:31:03.905388 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a357c7d-dbcf-41ac-b0cf-a77bd7b07651 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d5f621006a7a4c9a833fcafb96166ef8)` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:31:03.907576 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a357c7d-dbcf-41ac-b0cf-a77bd7b07651 admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:31:03.995497 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-7a357c7d-dbcf-41ac-b0cf-a77bd7b07651 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd5f621006a7a4c9a833fcafb96166ef8'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:31:04.008092 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-7a357c7d-dbcf-41ac-b0cf-a77bd7b07651 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd5f621006a7a4c9a833fcafb96166ef8'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:31:04.024265 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-7a357c7d-dbcf-41ac-b0cf-a77bd7b07651 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd5f621006a7a4c9a833fcafb96166ef8'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:31:04.025160 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-7a357c7d-dbcf-41ac-b0cf-a77bd7b07651 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd5f621006a7a4c9a833fcafb96166ef8'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:31:04.038917 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-7a357c7d-dbcf-41ac-b0cf-a77bd7b07651 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd5f621006a7a4c9a833fcafb96166ef8'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:31:04.049782 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1323/2617] 10.222.0.15 () {58 vars in 1224 bytes} [Thu Sep 15 12:31:03 2022] DELETE /identity/v3/users/d5f621006a7a4c9a833fcafb96166ef8 => generated 0 bytes in 394 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:31:04.052547 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08228dd8-3e2f-41f9-b149-ebc91a82b7c0 None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:31:04.053066 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08228dd8-3e2f-41f9-b149-ebc91a82b7c0 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:31:04.053448 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08228dd8-3e2f-41f9-b149-ebc91a82b7c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:31:04.075227 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-08228dd8-3e2f-41f9-b149-ebc91a82b7c0 None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:31:04.076784 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-08228dd8-3e2f-41f9-b149-ebc91a82b7c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:31:04.172743 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1295/2618] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 12:31:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:31:04.185162 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a357c7d-dbcf-41ac-b0cf-a77bd7b07651 admin admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:31:04.186052 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-7a357c7d-dbcf-41ac-b0cf-a77bd7b07651 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:31:04.235629 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:31:04.271411 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a357c7d-dbcf-41ac-b0cf-a77bd7b07651 admin admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:31:04.275012 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653a3332-b139-4717-9483-f5339717adcb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfbb4d6809064f9cb9e06626ed7d6814', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '540a32ef88764e8c8d9201ae22f6ddaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:31:04.275627 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653a3332-b139-4717-9483-f5339717adcb service neutron] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:31:04.275902 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653a3332-b139-4717-9483-f5339717adcb service neutron] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:31:04.276171 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653a3332-b139-4717-9483-f5339717adcb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:31:04.287439 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653a3332-b139-4717-9483-f5339717adcb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:31:04.288932 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653a3332-b139-4717-9483-f5339717adcb service neutron] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:31:04.301319 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1324/2619] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 12:31:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 12:31:04.753332 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08228dd8-3e2f-41f9-b149-ebc91a82b7c0 None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:31:04.756601 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-08228dd8-3e2f-41f9-b149-ebc91a82b7c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:31:04.797953 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08228dd8-3e2f-41f9-b149-ebc91a82b7c0 None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:31:04.809294 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27be971f-194e-42c9-b475-51a8fa7f62ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:31:04.811722 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27be971f-194e-42c9-b475-51a8fa7f62ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:31:04.812503 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27be971f-194e-42c9-b475-51a8fa7f62ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:31:04.813357 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27be971f-194e-42c9-b475-51a8fa7f62ab admin admin] PATH_INFO: `/v3/projects/a8ba2235ee22421c9aaa54700026038d` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:31:04.817633 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27be971f-194e-42c9-b475-51a8fa7f62ab admin admin] RBAC: Authorizing `identity:delete_project(project_id=a8ba2235ee22421c9aaa54700026038d)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:31:04.821538 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27be971f-194e-42c9-b475-51a8fa7f62ab admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:31:04.867806 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:31:04.924742 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-27be971f-194e-42c9-b475-51a8fa7f62ab admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a8ba2235ee22421c9aaa54700026038d', 'request_id': 'req-27be971f-194e-42c9-b475-51a8fa7f62ab'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:31:04.939974 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-27be971f-194e-42c9-b475-51a8fa7f62ab admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a8ba2235ee22421c9aaa54700026038d', 'request_id': 'req-27be971f-194e-42c9-b475-51a8fa7f62ab'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:31:04.981447 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-27be971f-194e-42c9-b475-51a8fa7f62ab admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a8ba2235ee22421c9aaa54700026038d', 'request_id': 'req-27be971f-194e-42c9-b475-51a8fa7f62ab'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:31:05.012214 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-27be971f-194e-42c9-b475-51a8fa7f62ab admin admin] The token cache is being invalidate because project a8ba2235ee22421c9aaa54700026038d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81448) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 12:31:05.013153 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-27be971f-194e-42c9-b475-51a8fa7f62ab admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:31:05.014781 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-27be971f-194e-42c9-b475-51a8fa7f62ab admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:31:05.025543 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653a3332-b139-4717-9483-f5339717adcb service neutron] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:31:05.032325 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1296/2620] 10.222.0.15 () {58 vars in 1233 bytes} [Thu Sep 15 12:31:04 2022] DELETE /identity/v3/projects/a8ba2235ee22421c9aaa54700026038d => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:31:05.040834 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653a3332-b139-4717-9483-f5339717adcb service neutron] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:31:05.122758 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628bbec8-a25d-4fc8-a09d-866e0b7aaf32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:31:05.123535 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628bbec8-a25d-4fc8-a09d-866e0b7aaf32 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:31:05.123862 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628bbec8-a25d-4fc8-a09d-866e0b7aaf32 admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:31:05.124209 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628bbec8-a25d-4fc8-a09d-866e0b7aaf32 admin admin] PATH_INFO: `/v3/users/538910d4b61e48f885827f7f56d1c23b` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:31:05.125688 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-628bbec8-a25d-4fc8-a09d-866e0b7aaf32 admin admin] RBAC: Authorizing `identity:delete_user(user_id=538910d4b61e48f885827f7f56d1c23b)` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:31:05.128637 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-628bbec8-a25d-4fc8-a09d-866e0b7aaf32 admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:31:05.194425 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-628bbec8-a25d-4fc8-a09d-866e0b7aaf32 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '538910d4b61e48f885827f7f56d1c23b'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:31:05.205970 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-628bbec8-a25d-4fc8-a09d-866e0b7aaf32 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '538910d4b61e48f885827f7f56d1c23b'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:31:05.222793 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-628bbec8-a25d-4fc8-a09d-866e0b7aaf32 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '538910d4b61e48f885827f7f56d1c23b'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:31:05.223709 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-628bbec8-a25d-4fc8-a09d-866e0b7aaf32 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '538910d4b61e48f885827f7f56d1c23b'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:31:05.232060 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-628bbec8-a25d-4fc8-a09d-866e0b7aaf32 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '538910d4b61e48f885827f7f56d1c23b'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:31:05.242441 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1325/2621] 10.222.0.15 () {58 vars in 1224 bytes} [Thu Sep 15 12:31:05 2022] DELETE /identity/v3/users/538910d4b61e48f885827f7f56d1c23b => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:31:05.535572 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27be971f-194e-42c9-b475-51a8fa7f62ab admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:31:05.537825 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-27be971f-194e-42c9-b475-51a8fa7f62ab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:31:05.553501 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27be971f-194e-42c9-b475-51a8fa7f62ab admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:31:05.630081 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa886fd-d75c-48cd-a69f-505720c5e93f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:31:05.631223 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa886fd-d75c-48cd-a69f-505720c5e93f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:31:05.631741 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa886fd-d75c-48cd-a69f-505720c5e93f admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:31:05.632247 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa886fd-d75c-48cd-a69f-505720c5e93f admin admin] PATH_INFO: `/v3/users/d6213a6ab5e4490cb8fd0372a7a8b495` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:31:05.634541 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa886fd-d75c-48cd-a69f-505720c5e93f admin admin] RBAC: Authorizing `identity:delete_user(user_id=d6213a6ab5e4490cb8fd0372a7a8b495)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:31:05.636809 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa886fd-d75c-48cd-a69f-505720c5e93f admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:31:05.714506 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-1aa886fd-d75c-48cd-a69f-505720c5e93f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd6213a6ab5e4490cb8fd0372a7a8b495'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:31:05.739117 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-1aa886fd-d75c-48cd-a69f-505720c5e93f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd6213a6ab5e4490cb8fd0372a7a8b495'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:31:05.748922 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-1aa886fd-d75c-48cd-a69f-505720c5e93f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd6213a6ab5e4490cb8fd0372a7a8b495'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:31:05.749600 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-1aa886fd-d75c-48cd-a69f-505720c5e93f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd6213a6ab5e4490cb8fd0372a7a8b495'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:31:05.761599 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-1aa886fd-d75c-48cd-a69f-505720c5e93f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd6213a6ab5e4490cb8fd0372a7a8b495'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:31:05.782867 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1297/2622] 10.222.0.15 () {58 vars in 1224 bytes} [Thu Sep 15 12:31:05 2022] DELETE /identity/v3/users/d6213a6ab5e4490cb8fd0372a7a8b495 => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:31:05.789284 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd04f81d-bed3-4c8a-affa-7cacc25df5d9 None None] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:31:05.789774 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd04f81d-bed3-4c8a-affa-7cacc25df5d9 None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:31:05.790248 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd04f81d-bed3-4c8a-affa-7cacc25df5d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:31:05.818307 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.auth.core [None req-dd04f81d-bed3-4c8a-affa-7cacc25df5d9 None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81449) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:31:05.820094 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-dd04f81d-bed3-4c8a-affa-7cacc25df5d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:31:05.894894 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:31:05.921784 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1326/2623] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 12:31:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:31:05.937259 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa886fd-d75c-48cd-a69f-505720c5e93f admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:31:05.939048 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-1aa886fd-d75c-48cd-a69f-505720c5e93f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:31:05.956533 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd04f81d-bed3-4c8a-affa-7cacc25df5d9 None None] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:31:05.961966 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-dd04f81d-bed3-4c8a-affa-7cacc25df5d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:31:06.008159 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:31:06.010458 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd04f81d-bed3-4c8a-affa-7cacc25df5d9 None None] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:31:06.017416 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1ba79c-bfcf-4ede-ad94-06c40a747601 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:31:06.018196 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1ba79c-bfcf-4ede-ad94-06c40a747601 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:31:06.018630 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1ba79c-bfcf-4ede-ad94-06c40a747601 admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:31:06.019051 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1ba79c-bfcf-4ede-ad94-06c40a747601 admin admin] PATH_INFO: `/v3/projects/cca25c1a554b4f93bed4800e7037efb1` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:31:06.021237 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be1ba79c-bfcf-4ede-ad94-06c40a747601 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cca25c1a554b4f93bed4800e7037efb1)` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:31:06.023703 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be1ba79c-bfcf-4ede-ad94-06c40a747601 admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:31:06.045619 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa886fd-d75c-48cd-a69f-505720c5e93f admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:31:06.051836 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ab6e6b-51ce-4e18-8c44-e7b017c7716d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfbb4d6809064f9cb9e06626ed7d6814', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '540a32ef88764e8c8d9201ae22f6ddaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:31:06.052885 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ab6e6b-51ce-4e18-8c44-e7b017c7716d service neutron] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:31:06.053444 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ab6e6b-51ce-4e18-8c44-e7b017c7716d service neutron] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:31:06.053904 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ab6e6b-51ce-4e18-8c44-e7b017c7716d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:31:06.066660 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ab6e6b-51ce-4e18-8c44-e7b017c7716d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:31:06.068364 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ab6e6b-51ce-4e18-8c44-e7b017c7716d service neutron] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:31:06.096480 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1298/2624] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 12:31:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 12:31:06.096695 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-be1ba79c-bfcf-4ede-ad94-06c40a747601 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cca25c1a554b4f93bed4800e7037efb1', 'request_id': 'req-be1ba79c-bfcf-4ede-ad94-06c40a747601'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:31:06.108849 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-be1ba79c-bfcf-4ede-ad94-06c40a747601 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cca25c1a554b4f93bed4800e7037efb1', 'request_id': 'req-be1ba79c-bfcf-4ede-ad94-06c40a747601'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:31:06.122562 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-be1ba79c-bfcf-4ede-ad94-06c40a747601 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cca25c1a554b4f93bed4800e7037efb1', 'request_id': 'req-be1ba79c-bfcf-4ede-ad94-06c40a747601'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:31:06.140499 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-be1ba79c-bfcf-4ede-ad94-06c40a747601 admin admin] The token cache is being invalidate because project cca25c1a554b4f93bed4800e7037efb1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81449) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 12:31:06.141225 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-be1ba79c-bfcf-4ede-ad94-06c40a747601 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:31:06.142152 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-be1ba79c-bfcf-4ede-ad94-06c40a747601 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:31:06.145792 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1327/2625] 10.222.0.15 () {58 vars in 1233 bytes} [Thu Sep 15 12:31:05 2022] DELETE /identity/v3/projects/cca25c1a554b4f93bed4800e7037efb1 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 12:31:06.151880 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf990220-effb-4ab9-b155-8a406309abb8 None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:31:06.152165 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf990220-effb-4ab9-b155-8a406309abb8 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:31:06.152430 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf990220-effb-4ab9-b155-8a406309abb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:31:06.202020 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-bf990220-effb-4ab9-b155-8a406309abb8 None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:31:06.203292 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-bf990220-effb-4ab9-b155-8a406309abb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:31:06.316815 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1299/2626] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 12:31:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:31:06.325279 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95be69e5-8ba4-41b6-a951-01b4be094121 None None] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:31:06.325689 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95be69e5-8ba4-41b6-a951-01b4be094121 None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:31:06.327445 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95be69e5-8ba4-41b6-a951-01b4be094121 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:31:06.378587 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.auth.core [None req-95be69e5-8ba4-41b6-a951-01b4be094121 None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81449) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:31:06.379820 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-95be69e5-8ba4-41b6-a951-01b4be094121 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:31:06.404735 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1328/2627] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 12:31:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:31:06.410952 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf990220-effb-4ab9-b155-8a406309abb8 None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:31:06.428418 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf990220-effb-4ab9-b155-8a406309abb8 None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:31:06.442344 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa6b0ea-58ff-4c72-b9ab-d0162c300cd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:31:06.443785 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa6b0ea-58ff-4c72-b9ab-d0162c300cd5 admin admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:31:06.444071 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa6b0ea-58ff-4c72-b9ab-d0162c300cd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:31:06.444345 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa6b0ea-58ff-4c72-b9ab-d0162c300cd5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:31:06.449093 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa6b0ea-58ff-4c72-b9ab-d0162c300cd5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:31:06.449711 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa6b0ea-58ff-4c72-b9ab-d0162c300cd5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:31:06.451897 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa6b0ea-58ff-4c72-b9ab-d0162c300cd5 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:31:06.477141 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1300/2628] 10.222.0.15 () {58 vars in 1153 bytes} [Thu Sep 15 12:31:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:31:06.483435 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95be69e5-8ba4-41b6-a951-01b4be094121 None None] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:31:06.499037 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95be69e5-8ba4-41b6-a951-01b4be094121 None None] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:31:06.507768 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14bee8a-aa1b-4bfb-8cd4-8f55d6388bd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:31:06.508637 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14bee8a-aa1b-4bfb-8cd4-8f55d6388bd5 admin admin] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:31:06.509203 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14bee8a-aa1b-4bfb-8cd4-8f55d6388bd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:31:06.509599 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14bee8a-aa1b-4bfb-8cd4-8f55d6388bd5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:31:06.510534 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14bee8a-aa1b-4bfb-8cd4-8f55d6388bd5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:31:06.514654 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b14bee8a-aa1b-4bfb-8cd4-8f55d6388bd5 admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:31:06.580439 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1329/2629] 10.222.0.15 () {60 vars in 1153 bytes} [Thu Sep 15 12:31:06 2022] POST /identity/v3/projects => generated 383 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 12:31:06.586420 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa6b0ea-58ff-4c72-b9ab-d0162c300cd5 admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:31:06.601099 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa6b0ea-58ff-4c72-b9ab-d0162c300cd5 admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:31:06.672283 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14bee8a-aa1b-4bfb-8cd4-8f55d6388bd5 admin admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:31:06.676151 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-b14bee8a-aa1b-4bfb-8cd4-8f55d6388bd5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:31:06.730011 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b371b251-2776-4b0f-98a0-ac8ff4169381 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:31:06.731631 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b371b251-2776-4b0f-98a0-ac8ff4169381 admin admin] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:31:06.732198 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b371b251-2776-4b0f-98a0-ac8ff4169381 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:31:06.732647 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b371b251-2776-4b0f-98a0-ac8ff4169381 admin admin] PATH_INFO: `/v3/users` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:31:06.734010 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b371b251-2776-4b0f-98a0-ac8ff4169381 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:31:06.736295 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b371b251-2776-4b0f-98a0-ac8ff4169381 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:31:06.769918 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1301/2630] 10.222.0.15 () {60 vars in 1144 bytes} [Thu Sep 15 12:31:06 2022] POST /identity/v3/users => generated 292 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 12:31:06.771826 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b371b251-2776-4b0f-98a0-ac8ff4169381 admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:31:06.797067 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b371b251-2776-4b0f-98a0-ac8ff4169381 admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:31:06.806921 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14bee8a-aa1b-4bfb-8cd4-8f55d6388bd5 admin admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:31:06.809649 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb018ea-7568-48eb-bc76-72c81b4aa6fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:31:06.810323 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb018ea-7568-48eb-bc76-72c81b4aa6fc admin admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:31:06.810787 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb018ea-7568-48eb-bc76-72c81b4aa6fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:31:06.811150 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb018ea-7568-48eb-bc76-72c81b4aa6fc admin admin] PATH_INFO: `/v3/roles` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:31:06.811956 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb018ea-7568-48eb-bc76-72c81b4aa6fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:31:06.814108 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e95f860-5308-4c63-b35d-9b314454a829 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:31:06.814346 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb018ea-7568-48eb-bc76-72c81b4aa6fc admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:31:06.817290 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e95f860-5308-4c63-b35d-9b314454a829 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:31:06.817988 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e95f860-5308-4c63-b35d-9b314454a829 admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:31:06.818477 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e95f860-5308-4c63-b35d-9b314454a829 admin admin] PATH_INFO: `/v3/projects/72ff3a200bf143f58e265995ec418f58` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:31:06.821165 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e95f860-5308-4c63-b35d-9b314454a829 admin admin] RBAC: Authorizing `identity:delete_project(project_id=72ff3a200bf143f58e265995ec418f58)` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:31:06.823944 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e95f860-5308-4c63-b35d-9b314454a829 admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:31:06.837118 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1302/2631] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 12:31:06 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 12:31:06.841287 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb018ea-7568-48eb-bc76-72c81b4aa6fc admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:31:06.858244 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb018ea-7568-48eb-bc76-72c81b4aa6fc admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:31:06.866482 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b114e2ec-6a7b-45d0-939d-8458a3d39238 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:31:06.868204 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b114e2ec-6a7b-45d0-939d-8458a3d39238 admin admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:31:06.868688 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b114e2ec-6a7b-45d0-939d-8458a3d39238 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:31:06.869541 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b114e2ec-6a7b-45d0-939d-8458a3d39238 admin admin] PATH_INFO: `/v3/roles` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:31:06.870750 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b114e2ec-6a7b-45d0-939d-8458a3d39238 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:31:06.873652 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b114e2ec-6a7b-45d0-939d-8458a3d39238 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:31:06.894959 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1303/2632] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 12:31:06 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 12:31:06.897062 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b114e2ec-6a7b-45d0-939d-8458a3d39238 admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:31:06.914167 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b114e2ec-6a7b-45d0-939d-8458a3d39238 admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:31:06.915488 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-4e95f860-5308-4c63-b35d-9b314454a829 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '72ff3a200bf143f58e265995ec418f58', 'request_id': 'req-4e95f860-5308-4c63-b35d-9b314454a829'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:31:06.919916 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=86,keystone:DELETE=11,keystone:INSERT=3 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:31:06.929353 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-4e95f860-5308-4c63-b35d-9b314454a829 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '72ff3a200bf143f58e265995ec418f58', 'request_id': 'req-4e95f860-5308-4c63-b35d-9b314454a829'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:31:06.941118 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-4e95f860-5308-4c63-b35d-9b314454a829 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '72ff3a200bf143f58e265995ec418f58', 'request_id': 'req-4e95f860-5308-4c63-b35d-9b314454a829'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:31:06.951951 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:31:06.976508 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-4e95f860-5308-4c63-b35d-9b314454a829 admin admin] The token cache is being invalidate because project 72ff3a200bf143f58e265995ec418f58 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81449) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 12:31:06.976508 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-4e95f860-5308-4c63-b35d-9b314454a829 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:31:06.976508 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-4e95f860-5308-4c63-b35d-9b314454a829 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:31:06.976508 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1330/2633] 10.222.0.15 () {58 vars in 1233 bytes} [Thu Sep 15 12:31:06 2022] DELETE /identity/v3/projects/72ff3a200bf143f58e265995ec418f58 => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 12:31:06.980350 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e95f860-5308-4c63-b35d-9b314454a829 admin admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:31:06.981779 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-4e95f860-5308-4c63-b35d-9b314454a829 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:31:07.018113 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c07aaad-75d0-4d7f-817d-a842e04322f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:31:07.019523 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c07aaad-75d0-4d7f-817d-a842e04322f8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:31:07.020219 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c07aaad-75d0-4d7f-817d-a842e04322f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:31:07.020572 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c07aaad-75d0-4d7f-817d-a842e04322f8 admin admin] PATH_INFO: `/v3/projects/f091b37acf9946debe9076dbb5f6bf2c/users/43c9119ec41d41f28bd2ce311de47b3b/roles/0679d6628a25438d90dc4bdc0401eda2` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:31:07.039578 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c07aaad-75d0-4d7f-817d-a842e04322f8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f091b37acf9946debe9076dbb5f6bf2c, user_id=43c9119ec41d41f28bd2ce311de47b3b, role_id=0679d6628a25438d90dc4bdc0401eda2)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:31:07.044181 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c07aaad-75d0-4d7f-817d-a842e04322f8 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:31:07.064078 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1304/2634] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 15 12:31:06 2022] PUT /identity/v3/projects/f091b37acf9946debe9076dbb5f6bf2c/users/43c9119ec41d41f28bd2ce311de47b3b/roles/0679d6628a25438d90dc4bdc0401eda2 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:31:07.075762 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cad7e5a-4310-41aa-ac49-96c9ae41931b None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:31:07.076181 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cad7e5a-4310-41aa-ac49-96c9ae41931b None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:31:07.076627 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cad7e5a-4310-41aa-ac49-96c9ae41931b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:31:07.124031 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-2cad7e5a-4310-41aa-ac49-96c9ae41931b None None] MFA Rules not processed for user `43c9119ec41d41f28bd2ce311de47b3b`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:31:07.126163 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-2cad7e5a-4310-41aa-ac49-96c9ae41931b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:31:07.131200 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e95f860-5308-4c63-b35d-9b314454a829 admin admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:31:07.249065 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1305/2635] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 12:31:07 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 12:31:07.253696 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cad7e5a-4310-41aa-ac49-96c9ae41931b None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:31:07.254786 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01bec619-f288-4f2e-b640-efc9d3e00fb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:31:07.255326 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-2cad7e5a-4310-41aa-ac49-96c9ae41931b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:31:07.255496 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bec619-f288-4f2e-b640-efc9d3e00fb7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:31:07.255819 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bec619-f288-4f2e-b640-efc9d3e00fb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:31:07.256089 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bec619-f288-4f2e-b640-efc9d3e00fb7 admin admin] PATH_INFO: `/v3/users/9704bf1ba2424f22a5a7699f1ce33194` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:31:07.257753 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01bec619-f288-4f2e-b640-efc9d3e00fb7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9704bf1ba2424f22a5a7699f1ce33194)` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:31:07.259791 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01bec619-f288-4f2e-b640-efc9d3e00fb7 admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:31:07.278253 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cad7e5a-4310-41aa-ac49-96c9ae41931b None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:31:07.285848 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06ccc82-27b9-498d-8bcf-80c8ddf41b9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:31:07.286930 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06ccc82-27b9-498d-8bcf-80c8ddf41b9b admin admin] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:31:07.287314 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06ccc82-27b9-498d-8bcf-80c8ddf41b9b admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:31:07.287693 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06ccc82-27b9-498d-8bcf-80c8ddf41b9b admin admin] PATH_INFO: `/v3/projects` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:31:07.289559 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06ccc82-27b9-498d-8bcf-80c8ddf41b9b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:31:07.292153 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06ccc82-27b9-498d-8bcf-80c8ddf41b9b admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:31:07.339381 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1306/2636] 10.222.0.15 () {60 vars in 1153 bytes} [Thu Sep 15 12:31:07 2022] POST /identity/v3/projects => generated 381 bytes in 87 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 15 12:31:07.344557 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06ccc82-27b9-498d-8bcf-80c8ddf41b9b admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:31:07.349108 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-01bec619-f288-4f2e-b640-efc9d3e00fb7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9704bf1ba2424f22a5a7699f1ce33194'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:31:07.358923 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-01bec619-f288-4f2e-b640-efc9d3e00fb7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9704bf1ba2424f22a5a7699f1ce33194'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:31:07.375122 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-01bec619-f288-4f2e-b640-efc9d3e00fb7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9704bf1ba2424f22a5a7699f1ce33194'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:31:07.375990 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-01bec619-f288-4f2e-b640-efc9d3e00fb7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9704bf1ba2424f22a5a7699f1ce33194'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:31:07.387564 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-01bec619-f288-4f2e-b640-efc9d3e00fb7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9704bf1ba2424f22a5a7699f1ce33194'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:31:07.403821 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1331/2637] 10.222.0.15 () {58 vars in 1224 bytes} [Thu Sep 15 12:31:06 2022] DELETE /identity/v3/users/9704bf1ba2424f22a5a7699f1ce33194 => generated 0 bytes in 424 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 12:31:07.438036 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06ccc82-27b9-498d-8bcf-80c8ddf41b9b admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:31:07.468701 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:31:07.522138 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9692134c-cf7b-45b6-828d-f28790d8c3f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:31:07.522893 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9692134c-cf7b-45b6-828d-f28790d8c3f7 admin admin] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:31:07.523175 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9692134c-cf7b-45b6-828d-f28790d8c3f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:31:07.523736 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9692134c-cf7b-45b6-828d-f28790d8c3f7 admin admin] PATH_INFO: `/v3/users` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:31:07.524564 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9692134c-cf7b-45b6-828d-f28790d8c3f7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:31:07.527944 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9692134c-cf7b-45b6-828d-f28790d8c3f7 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:31:07.571720 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01bec619-f288-4f2e-b640-efc9d3e00fb7 admin admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:31:07.573455 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-01bec619-f288-4f2e-b640-efc9d3e00fb7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:31:07.581648 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1307/2638] 10.222.0.15 () {60 vars in 1144 bytes} [Thu Sep 15 12:31:07 2022] POST /identity/v3/users => generated 297 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 12:31:07.595035 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01bec619-f288-4f2e-b640-efc9d3e00fb7 admin admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:31:07.602055 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b2dc3c-c6ad-4f4e-9640-5352ebcce369 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:31:07.603143 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b2dc3c-c6ad-4f4e-9640-5352ebcce369 admin admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:31:07.603434 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b2dc3c-c6ad-4f4e-9640-5352ebcce369 admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:31:07.603698 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b2dc3c-c6ad-4f4e-9640-5352ebcce369 admin admin] PATH_INFO: `/v3/roles` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:31:07.604327 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b2dc3c-c6ad-4f4e-9640-5352ebcce369 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:31:07.606105 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b2dc3c-c6ad-4f4e-9640-5352ebcce369 admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:31:07.607905 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:31:07.636185 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1332/2639] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 12:31:07 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 15 12:31:07.637736 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9692134c-cf7b-45b6-828d-f28790d8c3f7 admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:31:07.654028 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9692134c-cf7b-45b6-828d-f28790d8c3f7 admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:31:07.660629 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728bb075-8e22-41e2-99fd-3cb26c8f9059 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:31:07.661622 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728bb075-8e22-41e2-99fd-3cb26c8f9059 admin admin] REQUEST_METHOD: `PUT` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:31:07.662028 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728bb075-8e22-41e2-99fd-3cb26c8f9059 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:31:07.662422 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-728bb075-8e22-41e2-99fd-3cb26c8f9059 admin admin] PATH_INFO: `/v3/projects/5b43e19db10246da9f78657123bbb10b/users/0b729b1a18a343b5a1d83ac61abe4354/roles/9c769b5c82884766b2c510b1f0f1364c` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:31:07.694424 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728bb075-8e22-41e2-99fd-3cb26c8f9059 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5b43e19db10246da9f78657123bbb10b, user_id=0b729b1a18a343b5a1d83ac61abe4354, role_id=9c769b5c82884766b2c510b1f0f1364c)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:31:07.700524 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-728bb075-8e22-41e2-99fd-3cb26c8f9059 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:31:07.747903 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1308/2640] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 15 12:31:07 2022] PUT /identity/v3/projects/5b43e19db10246da9f78657123bbb10b/users/0b729b1a18a343b5a1d83ac61abe4354/roles/9c769b5c82884766b2c510b1f0f1364c => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:31:07.754420 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b2dc3c-c6ad-4f4e-9640-5352ebcce369 admin admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:31:07.771798 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b2dc3c-c6ad-4f4e-9640-5352ebcce369 admin admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:31:07.885183 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a063a91d-43ec-43b0-8fac-120fd8f8c669 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:31:07.886025 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a063a91d-43ec-43b0-8fac-120fd8f8c669 admin admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:31:07.886467 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a063a91d-43ec-43b0-8fac-120fd8f8c669 admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:31:07.886944 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a063a91d-43ec-43b0-8fac-120fd8f8c669 admin admin] PATH_INFO: `/v3/roles` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:31:07.888045 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a063a91d-43ec-43b0-8fac-120fd8f8c669 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:31:07.891415 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a063a91d-43ec-43b0-8fac-120fd8f8c669 admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:31:07.892736 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728bb075-8e22-41e2-99fd-3cb26c8f9059 admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:31:07.898702 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-728bb075-8e22-41e2-99fd-3cb26c8f9059 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:31:07.909886 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1333/2641] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 12:31:07 2022] GET /identity/v3/roles => generated 1420 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 12:31:07.918682 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a063a91d-43ec-43b0-8fac-120fd8f8c669 admin admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:31:07.941490 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a063a91d-43ec-43b0-8fac-120fd8f8c669 admin admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:31:07.951687 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a076896-21fb-40a4-88ae-805179379fa9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:31:07.952667 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a076896-21fb-40a4-88ae-805179379fa9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:31:07.953604 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a076896-21fb-40a4-88ae-805179379fa9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:31:07.953967 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a076896-21fb-40a4-88ae-805179379fa9 admin admin] PATH_INFO: `/v3/projects/5b43e19db10246da9f78657123bbb10b/users/0b729b1a18a343b5a1d83ac61abe4354/roles/9c769b5c82884766b2c510b1f0f1364c` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:31:07.958525 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a076896-21fb-40a4-88ae-805179379fa9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5b43e19db10246da9f78657123bbb10b, user_id=0b729b1a18a343b5a1d83ac61abe4354, role_id=9c769b5c82884766b2c510b1f0f1364c)` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:31:07.963665 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a076896-21fb-40a4-88ae-805179379fa9 admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:31:07.967941 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-728bb075-8e22-41e2-99fd-3cb26c8f9059 admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:31:07.985453 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1334/2642] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 15 12:31:07 2022] PUT /identity/v3/projects/5b43e19db10246da9f78657123bbb10b/users/0b729b1a18a343b5a1d83ac61abe4354/roles/9c769b5c82884766b2c510b1f0f1364c => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:31:07.990415 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb9bd90-2693-4ef0-98fa-8c081111e569 None None] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:31:07.990716 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb9bd90-2693-4ef0-98fa-8c081111e569 None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:31:07.991000 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb9bd90-2693-4ef0-98fa-8c081111e569 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:31:08.029506 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.auth.core [None req-1bb9bd90-2693-4ef0-98fa-8c081111e569 None None] MFA Rules not processed for user `0b729b1a18a343b5a1d83ac61abe4354`. Rule list: `[]` (Enabled: `True`). {{(pid=81449) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:31:08.031546 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-1bb9bd90-2693-4ef0-98fa-8c081111e569 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:31:08.089602 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e40e17e-915e-4d1e-bebd-d975b30a39e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:31:08.090264 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e40e17e-915e-4d1e-bebd-d975b30a39e5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:31:08.090703 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e40e17e-915e-4d1e-bebd-d975b30a39e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:31:08.091271 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e40e17e-915e-4d1e-bebd-d975b30a39e5 admin admin] PATH_INFO: `/v3/projects/5f6af8f1c76c489388fce417e83b75b7` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:31:08.092480 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e40e17e-915e-4d1e-bebd-d975b30a39e5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5f6af8f1c76c489388fce417e83b75b7)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:31:08.095124 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e40e17e-915e-4d1e-bebd-d975b30a39e5 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:31:08.154587 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1335/2643] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 12:31:07 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:31:08.168495 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb9bd90-2693-4ef0-98fa-8c081111e569 None None] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:31:08.171160 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-1bb9bd90-2693-4ef0-98fa-8c081111e569 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:31:08.178301 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-1e40e17e-915e-4d1e-bebd-d975b30a39e5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5f6af8f1c76c489388fce417e83b75b7', 'request_id': 'req-1e40e17e-915e-4d1e-bebd-d975b30a39e5'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:31:08.198737 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-1e40e17e-915e-4d1e-bebd-d975b30a39e5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5f6af8f1c76c489388fce417e83b75b7', 'request_id': 'req-1e40e17e-915e-4d1e-bebd-d975b30a39e5'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:31:08.212039 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-1e40e17e-915e-4d1e-bebd-d975b30a39e5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5f6af8f1c76c489388fce417e83b75b7', 'request_id': 'req-1e40e17e-915e-4d1e-bebd-d975b30a39e5'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:31:08.228323 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-1e40e17e-915e-4d1e-bebd-d975b30a39e5 admin admin] The token cache is being invalidate because project 5f6af8f1c76c489388fce417e83b75b7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81448) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 12:31:08.228677 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-1e40e17e-915e-4d1e-bebd-d975b30a39e5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:31:08.229497 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-1e40e17e-915e-4d1e-bebd-d975b30a39e5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:31:08.231426 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1309/2644] 10.222.0.15 () {58 vars in 1233 bytes} [Thu Sep 15 12:31:07 2022] DELETE /identity/v3/projects/5f6af8f1c76c489388fce417e83b75b7 => generated 0 bytes in 341 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:31:08.236643 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe39fed-ad9b-473c-a5bb-96288901c2a0 None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:31:08.237169 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe39fed-ad9b-473c-a5bb-96288901c2a0 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:31:08.238116 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe39fed-ad9b-473c-a5bb-96288901c2a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:31:08.263513 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb9bd90-2693-4ef0-98fa-8c081111e569 None None] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:31:08.270125 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-4fe39fed-ad9b-473c-a5bb-96288901c2a0 None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:31:08.271756 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-4fe39fed-ad9b-473c-a5bb-96288901c2a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:31:08.284369 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:31:08.293912 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:31:08.341590 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70d6c22-b245-4ed0-99d8-62627983ca5f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8191d3ccb874529b32f8d0026db629d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '540a32ef88764e8c8d9201ae22f6ddaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:31:08.342209 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70d6c22-b245-4ed0-99d8-62627983ca5f service cinder] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:31:08.342492 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70d6c22-b245-4ed0-99d8-62627983ca5f service cinder] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:31:08.342775 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70d6c22-b245-4ed0-99d8-62627983ca5f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:31:08.344723 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-e70d6c22-b245-4ed0-99d8-62627983ca5f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:31:08.385324 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1310/2645] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 12:31:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 12:31:08.386933 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc324ede-8ad6-4b5c-8292-d06222d17930 None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:31:08.388638 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc324ede-8ad6-4b5c-8292-d06222d17930 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:31:08.389019 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc324ede-8ad6-4b5c-8292-d06222d17930 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:31:08.418484 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-fc324ede-8ad6-4b5c-8292-d06222d17930 None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:31:08.419892 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-fc324ede-8ad6-4b5c-8292-d06222d17930 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:31:08.421356 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70d6c22-b245-4ed0-99d8-62627983ca5f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:31:08.422795 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70d6c22-b245-4ed0-99d8-62627983ca5f service cinder] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:31:08.433395 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1311/2646] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 12:31:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 12:31:08.434007 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1336/2647] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 12:31:08 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 266 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 15 12:31:08.435388 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc324ede-8ad6-4b5c-8292-d06222d17930 None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:31:08.448421 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc324ede-8ad6-4b5c-8292-d06222d17930 None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:31:08.468199 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce40fef1-34e6-4fbf-88bd-16531ed713e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:31:08.470329 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce40fef1-34e6-4fbf-88bd-16531ed713e7 admin admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:31:08.470834 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce40fef1-34e6-4fbf-88bd-16531ed713e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:31:08.472936 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce40fef1-34e6-4fbf-88bd-16531ed713e7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:31:08.474070 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce40fef1-34e6-4fbf-88bd-16531ed713e7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:31:08.474618 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce40fef1-34e6-4fbf-88bd-16531ed713e7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:31:08.476878 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce40fef1-34e6-4fbf-88bd-16531ed713e7 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:31:08.507332 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1312/2648] 10.222.0.15 () {58 vars in 1153 bytes} [Thu Sep 15 12:31:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:31:08.509077 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70d6c22-b245-4ed0-99d8-62627983ca5f service cinder] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:31:08.521641 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70d6c22-b245-4ed0-99d8-62627983ca5f service cinder] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:31:08.526509 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3714a67-9b06-4c5b-92d0-25e5c6dc79ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:31:08.527135 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3714a67-9b06-4c5b-92d0-25e5c6dc79ab admin admin] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:31:08.527420 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3714a67-9b06-4c5b-92d0-25e5c6dc79ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:31:08.527693 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3714a67-9b06-4c5b-92d0-25e5c6dc79ab admin admin] PATH_INFO: `/v3/projects` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:31:08.528343 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3714a67-9b06-4c5b-92d0-25e5c6dc79ab admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:31:08.530494 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3714a67-9b06-4c5b-92d0-25e5c6dc79ab admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:31:08.569216 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce40fef1-34e6-4fbf-88bd-16531ed713e7 admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:31:08.572397 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-ce40fef1-34e6-4fbf-88bd-16531ed713e7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:31:08.578248 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1337/2649] 10.222.0.15 () {60 vars in 1153 bytes} [Thu Sep 15 12:31:08 2022] POST /identity/v3/projects => generated 371 bytes in 70 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 12:31:08.584320 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3714a67-9b06-4c5b-92d0-25e5c6dc79ab admin admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:31:08.586802 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce40fef1-34e6-4fbf-88bd-16531ed713e7 admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:31:08.647004 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578ec112-e046-4b46-b7e2-e36539e63fda service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8191d3ccb874529b32f8d0026db629d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '540a32ef88764e8c8d9201ae22f6ddaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:31:08.648370 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578ec112-e046-4b46-b7e2-e36539e63fda service cinder] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:31:08.648843 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578ec112-e046-4b46-b7e2-e36539e63fda service cinder] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:31:08.649298 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578ec112-e046-4b46-b7e2-e36539e63fda service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:31:08.651268 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-578ec112-e046-4b46-b7e2-e36539e63fda service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:31:08.677297 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3714a67-9b06-4c5b-92d0-25e5c6dc79ab admin admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:31:08.684026 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c524e83c-f42e-4deb-9bec-117ff9fa929d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:31:08.684787 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c524e83c-f42e-4deb-9bec-117ff9fa929d admin admin] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:31:08.685652 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c524e83c-f42e-4deb-9bec-117ff9fa929d admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:31:08.686032 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c524e83c-f42e-4deb-9bec-117ff9fa929d admin admin] PATH_INFO: `/v3/users` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:31:08.686790 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c524e83c-f42e-4deb-9bec-117ff9fa929d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:31:08.692453 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c524e83c-f42e-4deb-9bec-117ff9fa929d admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:31:08.727615 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1338/2650] 10.222.0.15 () {60 vars in 1144 bytes} [Thu Sep 15 12:31:08 2022] POST /identity/v3/users => generated 286 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 12:31:08.731814 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c524e83c-f42e-4deb-9bec-117ff9fa929d admin admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:31:08.743525 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c524e83c-f42e-4deb-9bec-117ff9fa929d admin admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:31:08.747563 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578ec112-e046-4b46-b7e2-e36539e63fda service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:31:08.749035 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578ec112-e046-4b46-b7e2-e36539e63fda service cinder] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:31:08.751295 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216b3061-6295-46e8-be97-f7b02943f8d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:31:08.752091 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216b3061-6295-46e8-be97-f7b02943f8d8 admin admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:31:08.752697 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216b3061-6295-46e8-be97-f7b02943f8d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:31:08.753711 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216b3061-6295-46e8-be97-f7b02943f8d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:31:08.754499 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216b3061-6295-46e8-be97-f7b02943f8d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:31:08.758503 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216b3061-6295-46e8-be97-f7b02943f8d8 admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:31:08.772448 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1339/2651] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 12:31:08 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 12:31:08.775133 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1313/2652] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 12:31:08 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 206 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 15 12:31:08.776329 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216b3061-6295-46e8-be97-f7b02943f8d8 admin admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:31:08.792913 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216b3061-6295-46e8-be97-f7b02943f8d8 admin admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:31:08.799086 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a678426d-0838-4d41-b972-8ca405a9b458 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:31:08.799983 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a678426d-0838-4d41-b972-8ca405a9b458 admin admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:31:08.800529 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a678426d-0838-4d41-b972-8ca405a9b458 admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:31:08.801182 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a678426d-0838-4d41-b972-8ca405a9b458 admin admin] PATH_INFO: `/v3/roles` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:31:08.802166 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a678426d-0838-4d41-b972-8ca405a9b458 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:31:08.803887 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a678426d-0838-4d41-b972-8ca405a9b458 admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:31:08.817013 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1340/2653] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 12:31:08 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 12:31:08.822691 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578ec112-e046-4b46-b7e2-e36539e63fda service cinder] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:31:08.839447 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578ec112-e046-4b46-b7e2-e36539e63fda service cinder] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:31:08.850173 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a2f24e-0e39-45ba-bfb2-f3c83d9797d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:31:08.851093 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a2f24e-0e39-45ba-bfb2-f3c83d9797d8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:31:08.851613 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a2f24e-0e39-45ba-bfb2-f3c83d9797d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:31:08.851995 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a2f24e-0e39-45ba-bfb2-f3c83d9797d8 admin admin] PATH_INFO: `/v3/projects/0811b8305b9641aa884b0eb42a16e65b/users/c777718862d644e29b358ea28ad50cf7/roles/0679d6628a25438d90dc4bdc0401eda2` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:31:08.896692 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a2f24e-0e39-45ba-bfb2-f3c83d9797d8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0811b8305b9641aa884b0eb42a16e65b, user_id=c777718862d644e29b358ea28ad50cf7, role_id=0679d6628a25438d90dc4bdc0401eda2)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:31:08.912896 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a2f24e-0e39-45ba-bfb2-f3c83d9797d8 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:31:08.944629 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1314/2654] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 15 12:31:08 2022] PUT /identity/v3/projects/0811b8305b9641aa884b0eb42a16e65b/users/c777718862d644e29b358ea28ad50cf7/roles/0679d6628a25438d90dc4bdc0401eda2 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:31:08.951257 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea2c9f3-d689-488d-a30b-f088cab59164 None None] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:31:08.951952 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea2c9f3-d689-488d-a30b-f088cab59164 None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:31:08.952560 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea2c9f3-d689-488d-a30b-f088cab59164 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:31:08.996527 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.auth.core [None req-1ea2c9f3-d689-488d-a30b-f088cab59164 None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81449) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:31:08.998252 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-1ea2c9f3-d689-488d-a30b-f088cab59164 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:31:09.024886 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:31:09.151951 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1341/2655] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 12:31:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:31:09.161144 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a2f24e-0e39-45ba-bfb2-f3c83d9797d8 admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:31:09.163653 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-25a2f24e-0e39-45ba-bfb2-f3c83d9797d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:31:09.171268 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea2c9f3-d689-488d-a30b-f088cab59164 None None] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:31:09.172414 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-1ea2c9f3-d689-488d-a30b-f088cab59164 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:31:09.196575 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a2f24e-0e39-45ba-bfb2-f3c83d9797d8 admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:31:09.209419 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b270ddd-b182-4ba7-9fe2-e9b577813848 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:31:09.210477 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b270ddd-b182-4ba7-9fe2-e9b577813848 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:31:09.210794 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b270ddd-b182-4ba7-9fe2-e9b577813848 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:31:09.211168 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b270ddd-b182-4ba7-9fe2-e9b577813848 admin admin] PATH_INFO: `/v3/users/f1300202d61b4d908508a9a8035b7366` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:31:09.212675 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b270ddd-b182-4ba7-9fe2-e9b577813848 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f1300202d61b4d908508a9a8035b7366)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:31:09.217323 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b270ddd-b182-4ba7-9fe2-e9b577813848 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:31:09.271432 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=95,keystone:INSERT=1,keystone:DELETE=4 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:31:09.319458 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea2c9f3-d689-488d-a30b-f088cab59164 None None] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:31:09.323200 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-6b270ddd-b182-4ba7-9fe2-e9b577813848 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f1300202d61b4d908508a9a8035b7366'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:31:09.333457 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f253d8f3-e268-4fdd-9c57-2f7dfcde8eea service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfbb4d6809064f9cb9e06626ed7d6814', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '540a32ef88764e8c8d9201ae22f6ddaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:31:09.335803 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f253d8f3-e268-4fdd-9c57-2f7dfcde8eea service neutron] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:31:09.336198 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f253d8f3-e268-4fdd-9c57-2f7dfcde8eea service neutron] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:31:09.339397 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f253d8f3-e268-4fdd-9c57-2f7dfcde8eea service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:31:09.342811 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-6b270ddd-b182-4ba7-9fe2-e9b577813848 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f1300202d61b4d908508a9a8035b7366'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:31:09.354647 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-6b270ddd-b182-4ba7-9fe2-e9b577813848 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f1300202d61b4d908508a9a8035b7366'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:31:09.355473 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-6b270ddd-b182-4ba7-9fe2-e9b577813848 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f1300202d61b4d908508a9a8035b7366'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:31:09.363744 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f253d8f3-e268-4fdd-9c57-2f7dfcde8eea service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:31:09.365648 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-6b270ddd-b182-4ba7-9fe2-e9b577813848 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f1300202d61b4d908508a9a8035b7366'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:31:09.366082 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f253d8f3-e268-4fdd-9c57-2f7dfcde8eea service neutron] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:31:09.374223 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1315/2656] 10.222.0.15 () {58 vars in 1224 bytes} [Thu Sep 15 12:31:09 2022] DELETE /identity/v3/users/f1300202d61b4d908508a9a8035b7366 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 12:31:09.379573 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326f2a96-ace3-4e29-bd50-a464a7de91ba None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:31:09.379959 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326f2a96-ace3-4e29-bd50-a464a7de91ba None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:31:09.380330 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326f2a96-ace3-4e29-bd50-a464a7de91ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:31:09.401390 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-326f2a96-ace3-4e29-bd50-a464a7de91ba None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:31:09.401850 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1342/2657] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 12:31:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 217 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 12:31:09.403379 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-326f2a96-ace3-4e29-bd50-a464a7de91ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:31:09.521378 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1316/2658] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 12:31:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:31:09.533336 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f253d8f3-e268-4fdd-9c57-2f7dfcde8eea service neutron] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:31:09.534955 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-f253d8f3-e268-4fdd-9c57-2f7dfcde8eea service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:31:09.559430 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f253d8f3-e268-4fdd-9c57-2f7dfcde8eea service neutron] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:31:09.636460 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70ebadd-67d9-4cbd-8d06-861493a9c47e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfbb4d6809064f9cb9e06626ed7d6814', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '540a32ef88764e8c8d9201ae22f6ddaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:31:09.638994 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70ebadd-67d9-4cbd-8d06-861493a9c47e service neutron] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:31:09.640290 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70ebadd-67d9-4cbd-8d06-861493a9c47e service neutron] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:31:09.641641 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70ebadd-67d9-4cbd-8d06-861493a9c47e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:31:09.659734 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70ebadd-67d9-4cbd-8d06-861493a9c47e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:31:09.661846 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70ebadd-67d9-4cbd-8d06-861493a9c47e service neutron] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:31:09.672902 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1343/2659] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 12:31:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 12:31:10.130181 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326f2a96-ace3-4e29-bd50-a464a7de91ba None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:31:10.132650 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-326f2a96-ace3-4e29-bd50-a464a7de91ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:31:10.199508 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326f2a96-ace3-4e29-bd50-a464a7de91ba None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:31:10.205548 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886cb723-b0ce-4c4f-bd7a-e35089cc1bf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:31:10.206853 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886cb723-b0ce-4c4f-bd7a-e35089cc1bf1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:31:10.207535 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886cb723-b0ce-4c4f-bd7a-e35089cc1bf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:31:10.208084 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886cb723-b0ce-4c4f-bd7a-e35089cc1bf1 admin admin] PATH_INFO: `/v3/projects/cfb6d88be2484e9eab6dac4e5d216efa` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:31:10.209600 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886cb723-b0ce-4c4f-bd7a-e35089cc1bf1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cfb6d88be2484e9eab6dac4e5d216efa)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:31:10.212828 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886cb723-b0ce-4c4f-bd7a-e35089cc1bf1 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:31:10.272823 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-886cb723-b0ce-4c4f-bd7a-e35089cc1bf1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cfb6d88be2484e9eab6dac4e5d216efa', 'request_id': 'req-886cb723-b0ce-4c4f-bd7a-e35089cc1bf1'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:31:10.283636 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-886cb723-b0ce-4c4f-bd7a-e35089cc1bf1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cfb6d88be2484e9eab6dac4e5d216efa', 'request_id': 'req-886cb723-b0ce-4c4f-bd7a-e35089cc1bf1'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:31:10.294328 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-886cb723-b0ce-4c4f-bd7a-e35089cc1bf1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cfb6d88be2484e9eab6dac4e5d216efa', 'request_id': 'req-886cb723-b0ce-4c4f-bd7a-e35089cc1bf1'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:31:10.316013 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-886cb723-b0ce-4c4f-bd7a-e35089cc1bf1 admin admin] The token cache is being invalidate because project cfb6d88be2484e9eab6dac4e5d216efa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81448) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 12:31:10.316946 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-886cb723-b0ce-4c4f-bd7a-e35089cc1bf1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:31:10.317718 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-886cb723-b0ce-4c4f-bd7a-e35089cc1bf1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:31:10.323267 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1317/2660] 10.222.0.15 () {58 vars in 1233 bytes} [Thu Sep 15 12:31:10 2022] DELETE /identity/v3/projects/cfb6d88be2484e9eab6dac4e5d216efa => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:31:10.324959 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70ebadd-67d9-4cbd-8d06-861493a9c47e service neutron] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:31:10.345936 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70ebadd-67d9-4cbd-8d06-861493a9c47e service neutron] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:31:10.460062 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b1a0b1-c3be-4a01-ac20-4bcc544ea5f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:31:10.461297 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b1a0b1-c3be-4a01-ac20-4bcc544ea5f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:31:10.461811 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b1a0b1-c3be-4a01-ac20-4bcc544ea5f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:31:10.462298 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b1a0b1-c3be-4a01-ac20-4bcc544ea5f6 admin admin] PATH_INFO: `/v3/users/4f3d25cec8b64e68abe3bc0d23242af1` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:31:10.463858 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b1a0b1-c3be-4a01-ac20-4bcc544ea5f6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4f3d25cec8b64e68abe3bc0d23242af1)` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:31:10.466466 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b1a0b1-c3be-4a01-ac20-4bcc544ea5f6 admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:31:10.470345 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:31:10.583783 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5baac728-a142-4b86-b691-eb3fc7cf160a None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:31:10.584328 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5baac728-a142-4b86-b691-eb3fc7cf160a None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:31:10.584827 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5baac728-a142-4b86-b691-eb3fc7cf160a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:31:10.611558 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-02b1a0b1-c3be-4a01-ac20-4bcc544ea5f6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4f3d25cec8b64e68abe3bc0d23242af1'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:31:10.627021 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-02b1a0b1-c3be-4a01-ac20-4bcc544ea5f6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4f3d25cec8b64e68abe3bc0d23242af1'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:31:10.651457 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-02b1a0b1-c3be-4a01-ac20-4bcc544ea5f6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4f3d25cec8b64e68abe3bc0d23242af1'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:31:10.652305 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-02b1a0b1-c3be-4a01-ac20-4bcc544ea5f6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4f3d25cec8b64e68abe3bc0d23242af1'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:31:10.662806 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-02b1a0b1-c3be-4a01-ac20-4bcc544ea5f6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4f3d25cec8b64e68abe3bc0d23242af1'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:31:10.676861 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-5baac728-a142-4b86-b691-eb3fc7cf160a None None] MFA Rules not processed for user `c777718862d644e29b358ea28ad50cf7`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:31:10.681329 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1344/2661] 10.222.0.15 () {58 vars in 1224 bytes} [Thu Sep 15 12:31:10 2022] DELETE /identity/v3/users/4f3d25cec8b64e68abe3bc0d23242af1 => generated 0 bytes in 354 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:31:10.681502 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-5baac728-a142-4b86-b691-eb3fc7cf160a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:31:10.701466 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:31:10.903160 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1318/2662] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 12:31:10 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 317 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:31:10.904594 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b1a0b1-c3be-4a01-ac20-4bcc544ea5f6 admin admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:31:10.906167 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-02b1a0b1-c3be-4a01-ac20-4bcc544ea5f6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:31:11.015231 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b1a0b1-c3be-4a01-ac20-4bcc544ea5f6 admin admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:31:11.032104 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9b4392-642c-46d1-810a-984235557faa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:31:11.033033 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9b4392-642c-46d1-810a-984235557faa admin admin] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:31:11.033323 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9b4392-642c-46d1-810a-984235557faa admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:31:11.033658 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9b4392-642c-46d1-810a-984235557faa admin admin] PATH_INFO: `/v3/projects` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:31:11.035230 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9b4392-642c-46d1-810a-984235557faa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:31:11.037466 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9b4392-642c-46d1-810a-984235557faa admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:31:11.075377 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5baac728-a142-4b86-b691-eb3fc7cf160a None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:31:11.077520 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-5baac728-a142-4b86-b691-eb3fc7cf160a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:31:11.079545 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1345/2663] 10.222.0.15 () {60 vars in 1153 bytes} [Thu Sep 15 12:31:10 2022] POST /identity/v3/projects => generated 373 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 12:31:11.080669 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9b4392-642c-46d1-810a-984235557faa admin admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:31:11.098241 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5baac728-a142-4b86-b691-eb3fc7cf160a None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:31:11.168854 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9b4392-642c-46d1-810a-984235557faa admin admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:31:11.179263 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55bb86f3-24ae-41b0-bd8e-05cfe531a2fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:31:11.180481 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bb86f3-24ae-41b0-bd8e-05cfe531a2fa admin admin] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:31:11.181082 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bb86f3-24ae-41b0-bd8e-05cfe531a2fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:31:11.181506 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bb86f3-24ae-41b0-bd8e-05cfe531a2fa admin admin] PATH_INFO: `/v3/users` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:31:11.182844 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55bb86f3-24ae-41b0-bd8e-05cfe531a2fa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:31:11.184910 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55bb86f3-24ae-41b0-bd8e-05cfe531a2fa admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:31:11.189479 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7cbd3b7-fbda-47c4-870f-edd0d69d7dfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:31:11.190128 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cbd3b7-fbda-47c4-870f-edd0d69d7dfd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:31:11.190434 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cbd3b7-fbda-47c4-870f-edd0d69d7dfd admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:31:11.190823 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cbd3b7-fbda-47c4-870f-edd0d69d7dfd admin admin] PATH_INFO: `/v3/projects/3b2b2ab214054db39da9cc7c6b97131d` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:31:11.191952 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7cbd3b7-fbda-47c4-870f-edd0d69d7dfd admin admin] RBAC: Authorizing `identity:delete_project(project_id=3b2b2ab214054db39da9cc7c6b97131d)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:31:11.193997 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7cbd3b7-fbda-47c4-870f-edd0d69d7dfd admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:31:11.201281 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=85,keystone:DELETE=10,keystone:INSERT=5 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:31:11.225691 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1346/2664] 10.222.0.15 () {60 vars in 1144 bytes} [Thu Sep 15 12:31:11 2022] POST /identity/v3/users => generated 293 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 12:31:11.226442 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55bb86f3-24ae-41b0-bd8e-05cfe531a2fa admin admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:31:11.239817 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55bb86f3-24ae-41b0-bd8e-05cfe531a2fa admin admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:31:11.257314 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-d7cbd3b7-fbda-47c4-870f-edd0d69d7dfd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3b2b2ab214054db39da9cc7c6b97131d', 'request_id': 'req-d7cbd3b7-fbda-47c4-870f-edd0d69d7dfd'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:31:11.271330 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-d7cbd3b7-fbda-47c4-870f-edd0d69d7dfd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3b2b2ab214054db39da9cc7c6b97131d', 'request_id': 'req-d7cbd3b7-fbda-47c4-870f-edd0d69d7dfd'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:31:11.279287 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-d7cbd3b7-fbda-47c4-870f-edd0d69d7dfd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3b2b2ab214054db39da9cc7c6b97131d', 'request_id': 'req-d7cbd3b7-fbda-47c4-870f-edd0d69d7dfd'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:31:11.301836 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-d7cbd3b7-fbda-47c4-870f-edd0d69d7dfd admin admin] The token cache is being invalidate because project 3b2b2ab214054db39da9cc7c6b97131d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81448) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 12:31:11.302699 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-d7cbd3b7-fbda-47c4-870f-edd0d69d7dfd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:31:11.304141 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-d7cbd3b7-fbda-47c4-870f-edd0d69d7dfd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:31:11.307905 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1319/2665] 10.222.0.15 () {58 vars in 1233 bytes} [Thu Sep 15 12:31:11 2022] DELETE /identity/v3/projects/3b2b2ab214054db39da9cc7c6b97131d => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 12:31:11.313381 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ed57e1-b559-4943-b292-29ce1f579599 None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:31:11.313871 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ed57e1-b559-4943-b292-29ce1f579599 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:31:11.314428 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ed57e1-b559-4943-b292-29ce1f579599 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:31:11.336156 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8280d5-d150-41c0-a867-324a85c150f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:31:11.337095 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8280d5-d150-41c0-a867-324a85c150f6 admin admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:31:11.337522 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8280d5-d150-41c0-a867-324a85c150f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:31:11.337798 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8280d5-d150-41c0-a867-324a85c150f6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:31:11.338444 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8280d5-d150-41c0-a867-324a85c150f6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:31:11.340241 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8280d5-d150-41c0-a867-324a85c150f6 admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:31:11.346018 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-e1ed57e1-b559-4943-b292-29ce1f579599 None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:31:11.347723 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-e1ed57e1-b559-4943-b292-29ce1f579599 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:31:11.356944 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1347/2666] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 12:31:11 2022] GET /identity/v3/roles => generated 1420 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 12:31:11.361288 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8280d5-d150-41c0-a867-324a85c150f6 admin admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:31:11.362946 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1320/2667] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 12:31:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 12:31:11.363092 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-6e8280d5-d150-41c0-a867-324a85c150f6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:31:11.382046 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8280d5-d150-41c0-a867-324a85c150f6 admin admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:31:11.388276 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96009047-877d-47a9-97ed-64043043d938 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:31:11.389257 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96009047-877d-47a9-97ed-64043043d938 admin admin] REQUEST_METHOD: `PUT` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:31:11.389580 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96009047-877d-47a9-97ed-64043043d938 admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:31:11.389879 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96009047-877d-47a9-97ed-64043043d938 admin admin] PATH_INFO: `/v3/projects/fe2c3e20b5f64a1599fccbf437c65000/users/e81c271044734a5bb543d19197bb4dd5/roles/9c769b5c82884766b2c510b1f0f1364c` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:31:11.414485 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96009047-877d-47a9-97ed-64043043d938 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fe2c3e20b5f64a1599fccbf437c65000, user_id=e81c271044734a5bb543d19197bb4dd5, role_id=9c769b5c82884766b2c510b1f0f1364c)` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:31:11.418054 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96009047-877d-47a9-97ed-64043043d938 admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:31:11.436254 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076b4e13-570b-480a-8a9e-3372c5620a7a None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:31:11.436617 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076b4e13-570b-480a-8a9e-3372c5620a7a None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:31:11.436935 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076b4e13-570b-480a-8a9e-3372c5620a7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:31:11.439027 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1348/2668] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 15 12:31:11 2022] PUT /identity/v3/projects/fe2c3e20b5f64a1599fccbf437c65000/users/e81c271044734a5bb543d19197bb4dd5/roles/9c769b5c82884766b2c510b1f0f1364c => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 12:31:11.443065 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96009047-877d-47a9-97ed-64043043d938 admin admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:31:11.464090 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-076b4e13-570b-480a-8a9e-3372c5620a7a None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:31:11.465407 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-076b4e13-570b-480a-8a9e-3372c5620a7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:31:11.509705 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:31:11.541352 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96009047-877d-47a9-97ed-64043043d938 admin admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:31:11.549941 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3b6271-2bce-4d68-a876-a013de5cdecd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:31:11.550521 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3b6271-2bce-4d68-a876-a013de5cdecd admin admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:31:11.550802 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3b6271-2bce-4d68-a876-a013de5cdecd admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:31:11.551138 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3b6271-2bce-4d68-a876-a013de5cdecd admin admin] PATH_INFO: `/v3/roles` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:31:11.551730 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3b6271-2bce-4d68-a876-a013de5cdecd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:31:11.553395 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3b6271-2bce-4d68-a876-a013de5cdecd admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:31:11.556810 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1321/2669] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 12:31:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 12:31:11.559700 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076b4e13-570b-480a-8a9e-3372c5620a7a None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:31:11.564994 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1349/2670] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 12:31:11 2022] GET /identity/v3/roles => generated 1420 bytes in 122 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 12:31:11.569363 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3b6271-2bce-4d68-a876-a013de5cdecd admin admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:31:11.579257 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076b4e13-570b-480a-8a9e-3372c5620a7a None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:31:11.580127 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3b6271-2bce-4d68-a876-a013de5cdecd admin admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:31:11.586186 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b29695-73d8-4870-87da-75ead95ce9ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:31:11.586984 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b29695-73d8-4870-87da-75ead95ce9ac admin admin] REQUEST_METHOD: `PUT` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:31:11.587318 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b29695-73d8-4870-87da-75ead95ce9ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:31:11.587671 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b29695-73d8-4870-87da-75ead95ce9ac admin admin] PATH_INFO: `/v3/projects/fe2c3e20b5f64a1599fccbf437c65000/users/e81c271044734a5bb543d19197bb4dd5/roles/9c769b5c82884766b2c510b1f0f1364c` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:31:11.589858 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b29695-73d8-4870-87da-75ead95ce9ac admin admin] RBAC: Authorizing `identity:create_grant(project_id=fe2c3e20b5f64a1599fccbf437c65000, user_id=e81c271044734a5bb543d19197bb4dd5, role_id=9c769b5c82884766b2c510b1f0f1364c)` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:31:11.591079 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2260142f-2904-4305-b2a0-a6848c53480d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:31:11.591711 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2260142f-2904-4305-b2a0-a6848c53480d admin admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:31:11.592140 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2260142f-2904-4305-b2a0-a6848c53480d admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:31:11.592415 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2260142f-2904-4305-b2a0-a6848c53480d admin admin] PATH_INFO: `/v3/domains` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:31:11.593102 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2260142f-2904-4305-b2a0-a6848c53480d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:31:11.593489 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b29695-73d8-4870-87da-75ead95ce9ac admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:31:11.596342 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2260142f-2904-4305-b2a0-a6848c53480d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:31:11.598721 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2260142f-2904-4305-b2a0-a6848c53480d admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:31:11.607860 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1350/2671] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 15 12:31:11 2022] PUT /identity/v3/projects/fe2c3e20b5f64a1599fccbf437c65000/users/e81c271044734a5bb543d19197bb4dd5/roles/9c769b5c82884766b2c510b1f0f1364c => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 12:31:11.621578 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1322/2672] 10.222.0.15 () {58 vars in 1153 bytes} [Thu Sep 15 12:31:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Sep 15 12:31:11.627082 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b29695-73d8-4870-87da-75ead95ce9ac admin admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:31:11.709135 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:31:11.731199 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b29695-73d8-4870-87da-75ead95ce9ac admin admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:31:11.737604 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198fd8da-f662-4c1e-a2eb-f3d627bf43f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:31:11.738760 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198fd8da-f662-4c1e-a2eb-f3d627bf43f6 admin admin] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:31:11.739776 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198fd8da-f662-4c1e-a2eb-f3d627bf43f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:31:11.740328 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198fd8da-f662-4c1e-a2eb-f3d627bf43f6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:31:11.742153 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-198fd8da-f662-4c1e-a2eb-f3d627bf43f6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:31:11.744784 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-198fd8da-f662-4c1e-a2eb-f3d627bf43f6 admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:31:11.782946 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1351/2673] 10.222.0.15 () {60 vars in 1153 bytes} [Thu Sep 15 12:31:11 2022] POST /identity/v3/projects => generated 367 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 12:31:11.785361 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2260142f-2904-4305-b2a0-a6848c53480d admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:31:11.803461 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2260142f-2904-4305-b2a0-a6848c53480d admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:31:11.816706 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198fd8da-f662-4c1e-a2eb-f3d627bf43f6 admin admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:31:11.819025 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-198fd8da-f662-4c1e-a2eb-f3d627bf43f6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:31:11.939799 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f5a0d1-643f-4de6-8c07-45ad3766d0d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:31:11.941717 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f5a0d1-643f-4de6-8c07-45ad3766d0d0 admin admin] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:31:11.942012 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f5a0d1-643f-4de6-8c07-45ad3766d0d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:31:11.942289 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f5a0d1-643f-4de6-8c07-45ad3766d0d0 admin admin] PATH_INFO: `/v3/users` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:31:11.943484 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f5a0d1-643f-4de6-8c07-45ad3766d0d0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:31:11.946929 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f5a0d1-643f-4de6-8c07-45ad3766d0d0 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:31:11.986866 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198fd8da-f662-4c1e-a2eb-f3d627bf43f6 admin admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:31:11.993060 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1323/2674] 10.222.0.15 () {60 vars in 1144 bytes} [Thu Sep 15 12:31:11 2022] POST /identity/v3/users => generated 284 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 15 12:31:11.996252 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f5a0d1-643f-4de6-8c07-45ad3766d0d0 admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:31:12.001212 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f91e2a-496a-4293-929d-4d0a8400ee08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:31:12.001877 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f91e2a-496a-4293-929d-4d0a8400ee08 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:31:12.002364 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f91e2a-496a-4293-929d-4d0a8400ee08 admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:31:12.002643 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f91e2a-496a-4293-929d-4d0a8400ee08 admin admin] PATH_INFO: `/v3/users/43c9119ec41d41f28bd2ce311de47b3b` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:31:12.004424 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f91e2a-496a-4293-929d-4d0a8400ee08 admin admin] RBAC: Authorizing `identity:delete_user(user_id=43c9119ec41d41f28bd2ce311de47b3b)` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:31:12.007141 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f91e2a-496a-4293-929d-4d0a8400ee08 admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:31:12.037885 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f5a0d1-643f-4de6-8c07-45ad3766d0d0 admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:31:12.059840 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93188f29-915a-44c1-92f0-7078e59132eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:31:12.061390 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93188f29-915a-44c1-92f0-7078e59132eb admin admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:31:12.062951 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93188f29-915a-44c1-92f0-7078e59132eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:31:12.063714 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93188f29-915a-44c1-92f0-7078e59132eb admin admin] PATH_INFO: `/v3/roles` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:31:12.066221 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93188f29-915a-44c1-92f0-7078e59132eb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:31:12.071110 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93188f29-915a-44c1-92f0-7078e59132eb admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:31:12.095085 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-75f91e2a-496a-4293-929d-4d0a8400ee08 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '43c9119ec41d41f28bd2ce311de47b3b'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:31:12.101162 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1324/2675] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 12:31:11 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 12:31:12.104664 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93188f29-915a-44c1-92f0-7078e59132eb admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:31:12.107303 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-75f91e2a-496a-4293-929d-4d0a8400ee08 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '43c9119ec41d41f28bd2ce311de47b3b'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:31:12.123923 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-75f91e2a-496a-4293-929d-4d0a8400ee08 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '43c9119ec41d41f28bd2ce311de47b3b'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:31:12.124441 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93188f29-915a-44c1-92f0-7078e59132eb admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:31:12.124889 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-75f91e2a-496a-4293-929d-4d0a8400ee08 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '43c9119ec41d41f28bd2ce311de47b3b'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:31:12.134549 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-75f91e2a-496a-4293-929d-4d0a8400ee08 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '43c9119ec41d41f28bd2ce311de47b3b'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:31:12.141504 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201e90f3-8ef7-4053-9123-0232bf2e4cbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:31:12.142317 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201e90f3-8ef7-4053-9123-0232bf2e4cbd admin admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:31:12.145342 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1352/2676] 10.222.0.15 () {58 vars in 1224 bytes} [Thu Sep 15 12:31:11 2022] DELETE /identity/v3/users/43c9119ec41d41f28bd2ce311de47b3b => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 12:31:12.145506 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201e90f3-8ef7-4053-9123-0232bf2e4cbd admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:31:12.145506 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201e90f3-8ef7-4053-9123-0232bf2e4cbd admin admin] PATH_INFO: `/v3/roles` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:31:12.145506 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201e90f3-8ef7-4053-9123-0232bf2e4cbd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:31:12.146261 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201e90f3-8ef7-4053-9123-0232bf2e4cbd admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:31:12.150036 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a404b2-c80b-4d7a-9589-34e7f7765f8f None None] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:31:12.150329 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a404b2-c80b-4d7a-9589-34e7f7765f8f None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:31:12.151234 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a404b2-c80b-4d7a-9589-34e7f7765f8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:31:12.162671 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1325/2677] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 12:31:12 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 12:31:12.163718 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201e90f3-8ef7-4053-9123-0232bf2e4cbd admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:31:12.165823 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-201e90f3-8ef7-4053-9123-0232bf2e4cbd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:31:12.179076 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.auth.core [None req-46a404b2-c80b-4d7a-9589-34e7f7765f8f None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81449) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:31:12.180407 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-46a404b2-c80b-4d7a-9589-34e7f7765f8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:31:12.188805 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201e90f3-8ef7-4053-9123-0232bf2e4cbd admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:31:12.249221 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:31:12.278429 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1353/2678] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 12:31:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:31:12.285379 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c363e45b-5325-4341-a7ed-f8512a147dd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:31:12.286064 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c363e45b-5325-4341-a7ed-f8512a147dd7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:31:12.286340 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c363e45b-5325-4341-a7ed-f8512a147dd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:31:12.286606 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c363e45b-5325-4341-a7ed-f8512a147dd7 admin admin] PATH_INFO: `/v3/projects/e6064d75f39949199c149f111abcae66/users/e0c911f04b5f440896ded2c38b68b5d6/roles/0679d6628a25438d90dc4bdc0401eda2` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:31:12.292090 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a404b2-c80b-4d7a-9589-34e7f7765f8f None None] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:31:12.293606 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-46a404b2-c80b-4d7a-9589-34e7f7765f8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:31:12.317810 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c363e45b-5325-4341-a7ed-f8512a147dd7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e6064d75f39949199c149f111abcae66, user_id=e0c911f04b5f440896ded2c38b68b5d6, role_id=0679d6628a25438d90dc4bdc0401eda2)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:31:12.323693 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c363e45b-5325-4341-a7ed-f8512a147dd7 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:31:12.342604 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1326/2679] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 15 12:31:12 2022] PUT /identity/v3/projects/e6064d75f39949199c149f111abcae66/users/e0c911f04b5f440896ded2c38b68b5d6/roles/0679d6628a25438d90dc4bdc0401eda2 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:31:12.353298 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a752a508-f3d4-47b7-8130-f57bf9b50e1d None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:31:12.353598 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a752a508-f3d4-47b7-8130-f57bf9b50e1d None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:31:12.354008 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a752a508-f3d4-47b7-8130-f57bf9b50e1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:31:12.390098 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-a752a508-f3d4-47b7-8130-f57bf9b50e1d None None] MFA Rules not processed for user `e0c911f04b5f440896ded2c38b68b5d6`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:31:12.391846 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-a752a508-f3d4-47b7-8130-f57bf9b50e1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:31:12.392294 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a404b2-c80b-4d7a-9589-34e7f7765f8f None None] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:31:12.397023 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:31:12.461216 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f75daca-e5f8-4a9d-aced-35f523fc35ee service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfbb4d6809064f9cb9e06626ed7d6814', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '540a32ef88764e8c8d9201ae22f6ddaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:31:12.461941 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f75daca-e5f8-4a9d-aced-35f523fc35ee service neutron] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:31:12.462220 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f75daca-e5f8-4a9d-aced-35f523fc35ee service neutron] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:31:12.462496 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f75daca-e5f8-4a9d-aced-35f523fc35ee service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:31:12.550799 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1327/2680] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 12:31:12 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 12:31:12.555215 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3354002e-cb2a-4141-a37d-5102e808f979 None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:31:12.555215 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3354002e-cb2a-4141-a37d-5102e808f979 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:31:12.555215 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3354002e-cb2a-4141-a37d-5102e808f979 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:31:12.567310 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f75daca-e5f8-4a9d-aced-35f523fc35ee service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:31:12.568453 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f75daca-e5f8-4a9d-aced-35f523fc35ee service neutron] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:31:12.600699 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-3354002e-cb2a-4141-a37d-5102e808f979 None None] MFA Rules not processed for user `e81c271044734a5bb543d19197bb4dd5`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:31:12.601648 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1354/2681] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 12:31:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 12:31:12.602139 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-3354002e-cb2a-4141-a37d-5102e808f979 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:31:12.603019 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f75daca-e5f8-4a9d-aced-35f523fc35ee service neutron] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:31:12.605009 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-8f75daca-e5f8-4a9d-aced-35f523fc35ee service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:31:12.720373 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1328/2682] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 12:31:12 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:31:12.733322 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3354002e-cb2a-4141-a37d-5102e808f979 None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:31:12.735697 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-3354002e-cb2a-4141-a37d-5102e808f979 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:31:12.740060 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f75daca-e5f8-4a9d-aced-35f523fc35ee service neutron] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:31:12.743929 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d21659e-7fe4-4303-956e-b1ae1d4d588c service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3aed8f78903f44cda9e5254aa46cf2b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '540a32ef88764e8c8d9201ae22f6ddaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:31:12.745773 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d21659e-7fe4-4303-956e-b1ae1d4d588c service glance] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:31:12.747870 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d21659e-7fe4-4303-956e-b1ae1d4d588c service glance] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:31:12.748960 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d21659e-7fe4-4303-956e-b1ae1d4d588c service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:31:12.766071 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d21659e-7fe4-4303-956e-b1ae1d4d588c service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:31:12.768945 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d21659e-7fe4-4303-956e-b1ae1d4d588c service glance] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:31:12.779973 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1355/2683] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 12:31:12 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 12:31:12.832909 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3354002e-cb2a-4141-a37d-5102e808f979 None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:31:12.837164 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d345abc4-3fca-4c2e-afee-80b4f97ae82c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8191d3ccb874529b32f8d0026db629d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '540a32ef88764e8c8d9201ae22f6ddaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:31:12.837789 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d345abc4-3fca-4c2e-afee-80b4f97ae82c service cinder] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:31:12.838070 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d345abc4-3fca-4c2e-afee-80b4f97ae82c service cinder] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:31:12.838343 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d345abc4-3fca-4c2e-afee-80b4f97ae82c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:31:12.840199 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-d345abc4-3fca-4c2e-afee-80b4f97ae82c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:31:12.926345 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=100 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:31:12.943996 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d345abc4-3fca-4c2e-afee-80b4f97ae82c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:31:12.945672 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d345abc4-3fca-4c2e-afee-80b4f97ae82c service cinder] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:31:12.962663 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1329/2684] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 12:31:12 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 230 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 12:31:13.031064 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d21659e-7fe4-4303-956e-b1ae1d4d588c service glance] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:31:13.033386 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-3d21659e-7fe4-4303-956e-b1ae1d4d588c service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:31:13.058962 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=100 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:31:13.066195 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d21659e-7fe4-4303-956e-b1ae1d4d588c service glance] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:31:13.074731 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1acb5eb8-1214-42be-9f26-ff4bee8abae1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:31:13.075539 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acb5eb8-1214-42be-9f26-ff4bee8abae1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:31:13.076077 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acb5eb8-1214-42be-9f26-ff4bee8abae1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:31:13.076433 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acb5eb8-1214-42be-9f26-ff4bee8abae1 admin admin] PATH_INFO: `/v3/projects/f091b37acf9946debe9076dbb5f6bf2c` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:31:13.077808 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1acb5eb8-1214-42be-9f26-ff4bee8abae1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f091b37acf9946debe9076dbb5f6bf2c)` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:31:13.080152 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1acb5eb8-1214-42be-9f26-ff4bee8abae1 admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:31:13.153573 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-1acb5eb8-1214-42be-9f26-ff4bee8abae1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f091b37acf9946debe9076dbb5f6bf2c', 'request_id': 'req-1acb5eb8-1214-42be-9f26-ff4bee8abae1'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:31:13.162833 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-1acb5eb8-1214-42be-9f26-ff4bee8abae1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f091b37acf9946debe9076dbb5f6bf2c', 'request_id': 'req-1acb5eb8-1214-42be-9f26-ff4bee8abae1'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:31:13.170792 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-1acb5eb8-1214-42be-9f26-ff4bee8abae1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f091b37acf9946debe9076dbb5f6bf2c', 'request_id': 'req-1acb5eb8-1214-42be-9f26-ff4bee8abae1'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:31:13.185132 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-1acb5eb8-1214-42be-9f26-ff4bee8abae1 admin admin] The token cache is being invalidate because project f091b37acf9946debe9076dbb5f6bf2c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81449) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 12:31:13.185449 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-1acb5eb8-1214-42be-9f26-ff4bee8abae1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:31:13.186147 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-1acb5eb8-1214-42be-9f26-ff4bee8abae1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:31:13.188144 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1356/2685] 10.222.0.15 () {58 vars in 1233 bytes} [Thu Sep 15 12:31:13 2022] DELETE /identity/v3/projects/f091b37acf9946debe9076dbb5f6bf2c => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:31:13.193002 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d345abc4-3fca-4c2e-afee-80b4f97ae82c service cinder] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:31:13.220492 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d345abc4-3fca-4c2e-afee-80b4f97ae82c service cinder] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:31:13.330652 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1abd2af-8f85-4536-8de9-cac3348d2234 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:31:13.332349 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1abd2af-8f85-4536-8de9-cac3348d2234 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:31:13.332694 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1abd2af-8f85-4536-8de9-cac3348d2234 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:31:13.337234 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1abd2af-8f85-4536-8de9-cac3348d2234 admin admin] PATH_INFO: `/v3/users/0b729b1a18a343b5a1d83ac61abe4354` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:31:13.337234 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1abd2af-8f85-4536-8de9-cac3348d2234 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0b729b1a18a343b5a1d83ac61abe4354)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:31:13.337234 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1abd2af-8f85-4536-8de9-cac3348d2234 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:31:13.411583 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-f1abd2af-8f85-4536-8de9-cac3348d2234 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0b729b1a18a343b5a1d83ac61abe4354'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:31:13.451892 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-f1abd2af-8f85-4536-8de9-cac3348d2234 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0b729b1a18a343b5a1d83ac61abe4354'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:31:13.469421 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-f1abd2af-8f85-4536-8de9-cac3348d2234 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0b729b1a18a343b5a1d83ac61abe4354'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:31:13.470579 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-f1abd2af-8f85-4536-8de9-cac3348d2234 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0b729b1a18a343b5a1d83ac61abe4354'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:31:13.485192 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-f1abd2af-8f85-4536-8de9-cac3348d2234 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0b729b1a18a343b5a1d83ac61abe4354'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:31:13.496213 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1330/2686] 10.222.0.15 () {58 vars in 1224 bytes} [Thu Sep 15 12:31:13 2022] DELETE /identity/v3/users/0b729b1a18a343b5a1d83ac61abe4354 => generated 0 bytes in 304 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 12:31:13.873260 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1acb5eb8-1214-42be-9f26-ff4bee8abae1 admin admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:31:13.875267 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-1acb5eb8-1214-42be-9f26-ff4bee8abae1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:31:13.903674 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1acb5eb8-1214-42be-9f26-ff4bee8abae1 admin admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:31:14.001195 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a000f149-fd88-43f5-bb0f-04ee8fee5bfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:31:14.001899 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a000f149-fd88-43f5-bb0f-04ee8fee5bfe admin admin] REQUEST_METHOD: `DELETE` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:31:14.002173 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a000f149-fd88-43f5-bb0f-04ee8fee5bfe admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:31:14.002639 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a000f149-fd88-43f5-bb0f-04ee8fee5bfe admin admin] PATH_INFO: `/v3/projects/5b43e19db10246da9f78657123bbb10b` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:31:14.003763 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a000f149-fd88-43f5-bb0f-04ee8fee5bfe admin admin] RBAC: Authorizing `identity:delete_project(project_id=5b43e19db10246da9f78657123bbb10b)` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:31:14.005712 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a000f149-fd88-43f5-bb0f-04ee8fee5bfe admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:31:14.096201 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-a000f149-fd88-43f5-bb0f-04ee8fee5bfe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5b43e19db10246da9f78657123bbb10b', 'request_id': 'req-a000f149-fd88-43f5-bb0f-04ee8fee5bfe'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:31:14.104405 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-a000f149-fd88-43f5-bb0f-04ee8fee5bfe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5b43e19db10246da9f78657123bbb10b', 'request_id': 'req-a000f149-fd88-43f5-bb0f-04ee8fee5bfe'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:31:14.110454 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-a000f149-fd88-43f5-bb0f-04ee8fee5bfe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5b43e19db10246da9f78657123bbb10b', 'request_id': 'req-a000f149-fd88-43f5-bb0f-04ee8fee5bfe'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:31:14.122949 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-a000f149-fd88-43f5-bb0f-04ee8fee5bfe admin admin] The token cache is being invalidate because project 5b43e19db10246da9f78657123bbb10b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81449) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 12:31:14.123315 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-a000f149-fd88-43f5-bb0f-04ee8fee5bfe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:31:14.124209 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-a000f149-fd88-43f5-bb0f-04ee8fee5bfe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:31:14.125815 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1357/2687] 10.222.0.15 () {58 vars in 1233 bytes} [Thu Sep 15 12:31:13 2022] DELETE /identity/v3/projects/5b43e19db10246da9f78657123bbb10b => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 12:31:14.130779 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98d539c-e1a3-4b75-9155-c73330d0812b None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:31:14.131092 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98d539c-e1a3-4b75-9155-c73330d0812b None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:31:14.131323 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98d539c-e1a3-4b75-9155-c73330d0812b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:31:14.164655 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-a98d539c-e1a3-4b75-9155-c73330d0812b None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:31:14.166232 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-a98d539c-e1a3-4b75-9155-c73330d0812b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:31:14.242833 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1331/2688] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 12:31:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:31:14.250849 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68035cfc-6723-4c23-830a-1f92c1797257 None None] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:31:14.251217 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68035cfc-6723-4c23-830a-1f92c1797257 None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:31:14.251564 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68035cfc-6723-4c23-830a-1f92c1797257 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:31:14.276126 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.auth.core [None req-68035cfc-6723-4c23-830a-1f92c1797257 None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81449) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:31:14.277865 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-68035cfc-6723-4c23-830a-1f92c1797257 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:31:14.287551 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1358/2689] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 12:31:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:31:14.292108 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98d539c-e1a3-4b75-9155-c73330d0812b None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:31:14.298242 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:31:14.302758 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98d539c-e1a3-4b75-9155-c73330d0812b None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:31:14.309479 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3ba29f-6901-4982-b723-52da14083886 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:31:14.310519 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3ba29f-6901-4982-b723-52da14083886 admin admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:31:14.311140 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3ba29f-6901-4982-b723-52da14083886 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:31:14.311781 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3ba29f-6901-4982-b723-52da14083886 admin admin] PATH_INFO: `/v3/domains` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:31:14.312429 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3ba29f-6901-4982-b723-52da14083886 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:31:14.312729 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3ba29f-6901-4982-b723-52da14083886 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:31:14.314465 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3ba29f-6901-4982-b723-52da14083886 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:31:14.330328 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1332/2690] 10.222.0.15 () {58 vars in 1153 bytes} [Thu Sep 15 12:31:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:31:14.332157 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68035cfc-6723-4c23-830a-1f92c1797257 None None] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:31:14.342604 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68035cfc-6723-4c23-830a-1f92c1797257 None None] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:31:14.350521 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1589309a-4ef2-4d63-bf21-5afa4f940e24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:31:14.351142 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1589309a-4ef2-4d63-bf21-5afa4f940e24 admin admin] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:31:14.351466 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1589309a-4ef2-4d63-bf21-5afa4f940e24 admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:31:14.351704 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1589309a-4ef2-4d63-bf21-5afa4f940e24 admin admin] PATH_INFO: `/v3/projects` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:31:14.352397 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1589309a-4ef2-4d63-bf21-5afa4f940e24 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:31:14.354332 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1589309a-4ef2-4d63-bf21-5afa4f940e24 admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:31:14.364377 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:31:14.385475 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1359/2691] 10.222.0.15 () {60 vars in 1153 bytes} [Thu Sep 15 12:31:14 2022] POST /identity/v3/projects => generated 383 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 12:31:14.389591 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3ba29f-6901-4982-b723-52da14083886 admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:31:14.401310 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3ba29f-6901-4982-b723-52da14083886 admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:31:14.481474 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d7dc4f-ad39-427e-8e82-78a77b789099 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:31:14.482260 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d7dc4f-ad39-427e-8e82-78a77b789099 admin admin] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:31:14.482543 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d7dc4f-ad39-427e-8e82-78a77b789099 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:31:14.482809 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d7dc4f-ad39-427e-8e82-78a77b789099 admin admin] PATH_INFO: `/v3/users` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:31:14.483450 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d7dc4f-ad39-427e-8e82-78a77b789099 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:31:14.485746 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d7dc4f-ad39-427e-8e82-78a77b789099 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:31:14.519329 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1333/2692] 10.222.0.15 () {60 vars in 1144 bytes} [Thu Sep 15 12:31:14 2022] POST /identity/v3/users => generated 292 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 15 12:31:14.525548 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1589309a-4ef2-4d63-bf21-5afa4f940e24 admin admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:31:14.538053 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1589309a-4ef2-4d63-bf21-5afa4f940e24 admin admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:31:14.542885 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87494896-c9bc-4d4c-9804-daf1db5b2d3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:31:14.543572 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87494896-c9bc-4d4c-9804-daf1db5b2d3a admin admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:31:14.543925 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87494896-c9bc-4d4c-9804-daf1db5b2d3a admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:31:14.544270 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87494896-c9bc-4d4c-9804-daf1db5b2d3a admin admin] PATH_INFO: `/v3/roles` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:31:14.545019 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87494896-c9bc-4d4c-9804-daf1db5b2d3a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:31:14.546737 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87494896-c9bc-4d4c-9804-daf1db5b2d3a admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:31:14.563749 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1360/2693] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 12:31:14 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 12:31:14.569362 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d7dc4f-ad39-427e-8e82-78a77b789099 admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:31:14.581666 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d7dc4f-ad39-427e-8e82-78a77b789099 admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:31:14.587360 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552ab83e-7950-4ee8-be55-b849d53385fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:31:14.588145 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552ab83e-7950-4ee8-be55-b849d53385fb admin admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:31:14.588428 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552ab83e-7950-4ee8-be55-b849d53385fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:31:14.588961 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552ab83e-7950-4ee8-be55-b849d53385fb admin admin] PATH_INFO: `/v3/roles` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:31:14.589406 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-552ab83e-7950-4ee8-be55-b849d53385fb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:31:14.591049 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-552ab83e-7950-4ee8-be55-b849d53385fb admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:31:14.603610 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1334/2694] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 12:31:14 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 12:31:14.608398 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87494896-c9bc-4d4c-9804-daf1db5b2d3a admin admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:31:14.622604 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87494896-c9bc-4d4c-9804-daf1db5b2d3a admin admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:31:14.629675 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac3d036-f42c-4ace-93fa-0bf73489669b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:31:14.630470 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac3d036-f42c-4ace-93fa-0bf73489669b admin admin] REQUEST_METHOD: `PUT` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:31:14.630953 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac3d036-f42c-4ace-93fa-0bf73489669b admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:31:14.631281 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac3d036-f42c-4ace-93fa-0bf73489669b admin admin] PATH_INFO: `/v3/projects/9f94d0d282214ac8bc1f50d83f14f517/users/816b6ea10d8140d8be8de34b4a312224/roles/0679d6628a25438d90dc4bdc0401eda2` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:31:14.664574 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac3d036-f42c-4ace-93fa-0bf73489669b admin admin] RBAC: Authorizing `identity:create_grant(project_id=9f94d0d282214ac8bc1f50d83f14f517, user_id=816b6ea10d8140d8be8de34b4a312224, role_id=0679d6628a25438d90dc4bdc0401eda2)` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:31:14.670226 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac3d036-f42c-4ace-93fa-0bf73489669b admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:31:14.700841 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1361/2695] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 15 12:31:14 2022] PUT /identity/v3/projects/9f94d0d282214ac8bc1f50d83f14f517/users/816b6ea10d8140d8be8de34b4a312224/roles/0679d6628a25438d90dc4bdc0401eda2 => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 12:31:14.701061 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552ab83e-7950-4ee8-be55-b849d53385fb admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:31:14.701061 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-552ab83e-7950-4ee8-be55-b849d53385fb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:31:14.705828 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ca6a2b-3d26-4d51-acb0-3f18c3fb1efc None None] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:31:14.706122 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ca6a2b-3d26-4d51-acb0-3f18c3fb1efc None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:31:14.706398 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ca6a2b-3d26-4d51-acb0-3f18c3fb1efc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:31:14.758367 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.auth.core [None req-44ca6a2b-3d26-4d51-acb0-3f18c3fb1efc None None] MFA Rules not processed for user `816b6ea10d8140d8be8de34b4a312224`. Rule list: `[]` (Enabled: `True`). {{(pid=81449) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:31:14.760723 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-44ca6a2b-3d26-4d51-acb0-3f18c3fb1efc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:31:14.796883 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552ab83e-7950-4ee8-be55-b849d53385fb admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:31:14.855715 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814f2e0f-61bb-4d6c-a803-aed1b8b225ac service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8191d3ccb874529b32f8d0026db629d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '540a32ef88764e8c8d9201ae22f6ddaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:31:14.856349 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814f2e0f-61bb-4d6c-a803-aed1b8b225ac service cinder] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:31:14.856628 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814f2e0f-61bb-4d6c-a803-aed1b8b225ac service cinder] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:31:14.856944 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814f2e0f-61bb-4d6c-a803-aed1b8b225ac service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:31:14.859073 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-814f2e0f-61bb-4d6c-a803-aed1b8b225ac service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:31:14.866130 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1362/2696] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 12:31:14 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 12:31:14.871070 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ca6a2b-3d26-4d51-acb0-3f18c3fb1efc None None] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:31:14.912990 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:31:14.939053 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814f2e0f-61bb-4d6c-a803-aed1b8b225ac service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:31:14.940561 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814f2e0f-61bb-4d6c-a803-aed1b8b225ac service cinder] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:31:14.951408 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1335/2697] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 12:31:14 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 261 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 12:31:14.957187 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ca6a2b-3d26-4d51-acb0-3f18c3fb1efc None None] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:31:14.965834 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7806580a-965c-47b3-a837-51649353dbc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:31:14.967587 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7806580a-965c-47b3-a837-51649353dbc7 admin admin] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:31:14.968078 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7806580a-965c-47b3-a837-51649353dbc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:31:14.968499 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7806580a-965c-47b3-a837-51649353dbc7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:31:14.969549 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7806580a-965c-47b3-a837-51649353dbc7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:31:14.976937 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7806580a-965c-47b3-a837-51649353dbc7 admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:31:15.013639 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1363/2698] 10.222.0.15 () {60 vars in 1153 bytes} [Thu Sep 15 12:31:14 2022] POST /identity/v3/projects => generated 385 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 12:31:15.018000 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814f2e0f-61bb-4d6c-a803-aed1b8b225ac service cinder] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:31:15.029379 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814f2e0f-61bb-4d6c-a803-aed1b8b225ac service cinder] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:31:15.129539 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88089b8e-b7d8-44df-97ec-4454e09241ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:31:15.130368 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88089b8e-b7d8-44df-97ec-4454e09241ce admin admin] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:31:15.130747 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88089b8e-b7d8-44df-97ec-4454e09241ce admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:31:15.131295 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88089b8e-b7d8-44df-97ec-4454e09241ce admin admin] PATH_INFO: `/v3/users` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:31:15.132556 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88089b8e-b7d8-44df-97ec-4454e09241ce admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:31:15.135777 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88089b8e-b7d8-44df-97ec-4454e09241ce admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:31:15.160339 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1336/2699] 10.222.0.15 () {60 vars in 1144 bytes} [Thu Sep 15 12:31:15 2022] POST /identity/v3/users => generated 293 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 12:31:15.164595 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7806580a-965c-47b3-a837-51649353dbc7 admin admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:31:15.175992 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7806580a-965c-47b3-a837-51649353dbc7 admin admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:31:15.181020 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe87e29-01e3-4ddc-a408-1d5381a86b4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:31:15.181630 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe87e29-01e3-4ddc-a408-1d5381a86b4d admin admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:31:15.181907 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe87e29-01e3-4ddc-a408-1d5381a86b4d admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:31:15.182197 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe87e29-01e3-4ddc-a408-1d5381a86b4d admin admin] PATH_INFO: `/v3/roles` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:31:15.182817 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe87e29-01e3-4ddc-a408-1d5381a86b4d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:31:15.184388 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe87e29-01e3-4ddc-a408-1d5381a86b4d admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:31:15.194714 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1364/2700] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 12:31:15 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 12:31:15.198804 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88089b8e-b7d8-44df-97ec-4454e09241ce admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:31:15.210931 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88089b8e-b7d8-44df-97ec-4454e09241ce admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:31:15.216768 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87823f29-4ddb-4971-9214-af6e49ecc217 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:31:15.217386 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87823f29-4ddb-4971-9214-af6e49ecc217 admin admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:31:15.217710 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87823f29-4ddb-4971-9214-af6e49ecc217 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:31:15.217930 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87823f29-4ddb-4971-9214-af6e49ecc217 admin admin] PATH_INFO: `/v3/roles` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:31:15.218563 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87823f29-4ddb-4971-9214-af6e49ecc217 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:31:15.220132 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87823f29-4ddb-4971-9214-af6e49ecc217 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:31:15.230656 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1337/2701] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 12:31:15 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 12:31:15.234785 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe87e29-01e3-4ddc-a408-1d5381a86b4d admin admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:31:15.244704 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe87e29-01e3-4ddc-a408-1d5381a86b4d admin admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:31:15.249491 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b1abf0-7bb4-4b2f-af2d-fd8d2a37d8f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:31:15.250137 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b1abf0-7bb4-4b2f-af2d-fd8d2a37d8f3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:31:15.250412 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b1abf0-7bb4-4b2f-af2d-fd8d2a37d8f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:31:15.250686 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0b1abf0-7bb4-4b2f-af2d-fd8d2a37d8f3 admin admin] PATH_INFO: `/v3/projects/b1d8e76a968b42cfa496700755b49a4f/users/f9f10b0cb37d4eb5acf3e84df8e196a6/roles/0679d6628a25438d90dc4bdc0401eda2` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:31:15.265826 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b1abf0-7bb4-4b2f-af2d-fd8d2a37d8f3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b1d8e76a968b42cfa496700755b49a4f, user_id=f9f10b0cb37d4eb5acf3e84df8e196a6, role_id=0679d6628a25438d90dc4bdc0401eda2)` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:31:15.269227 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0b1abf0-7bb4-4b2f-af2d-fd8d2a37d8f3 admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:31:15.272305 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:31:15.283700 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1365/2702] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 15 12:31:15 2022] PUT /identity/v3/projects/b1d8e76a968b42cfa496700755b49a4f/users/f9f10b0cb37d4eb5acf3e84df8e196a6/roles/0679d6628a25438d90dc4bdc0401eda2 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 12:31:15.295964 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0645dcb-148b-4f0f-9fa5-b59435a7a28d None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:31:15.296255 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0645dcb-148b-4f0f-9fa5-b59435a7a28d None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:31:15.296529 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0645dcb-148b-4f0f-9fa5-b59435a7a28d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:31:15.325211 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-f0645dcb-148b-4f0f-9fa5-b59435a7a28d None None] MFA Rules not processed for user `f9f10b0cb37d4eb5acf3e84df8e196a6`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:31:15.326424 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-f0645dcb-148b-4f0f-9fa5-b59435a7a28d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:31:15.391261 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1338/2703] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 12:31:15 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:31:15.396907 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b1abf0-7bb4-4b2f-af2d-fd8d2a37d8f3 admin admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:31:15.460818 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0b1abf0-7bb4-4b2f-af2d-fd8d2a37d8f3 admin admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:31:15.465627 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10235997-74d0-45a3-88b2-bb2d520b7fd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:31:15.466237 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10235997-74d0-45a3-88b2-bb2d520b7fd4 admin admin] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:31:15.466534 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10235997-74d0-45a3-88b2-bb2d520b7fd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:31:15.466806 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10235997-74d0-45a3-88b2-bb2d520b7fd4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:31:15.467530 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10235997-74d0-45a3-88b2-bb2d520b7fd4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:31:15.469562 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10235997-74d0-45a3-88b2-bb2d520b7fd4 admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:31:15.498212 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1366/2704] 10.222.0.15 () {60 vars in 1153 bytes} [Thu Sep 15 12:31:15 2022] POST /identity/v3/projects => generated 383 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 15 12:31:15.502389 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0645dcb-148b-4f0f-9fa5-b59435a7a28d None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:31:15.517485 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0645dcb-148b-4f0f-9fa5-b59435a7a28d None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:31:15.561513 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:31:15.579351 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d5eeab-ef4f-4243-bc3b-34272e7b1af7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:31:15.579973 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d5eeab-ef4f-4243-bc3b-34272e7b1af7 admin admin] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:31:15.580256 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d5eeab-ef4f-4243-bc3b-34272e7b1af7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:31:15.580530 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d5eeab-ef4f-4243-bc3b-34272e7b1af7 admin admin] PATH_INFO: `/v3/users` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:31:15.581782 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d5eeab-ef4f-4243-bc3b-34272e7b1af7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:31:15.583714 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d5eeab-ef4f-4243-bc3b-34272e7b1af7 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:31:15.604997 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1339/2705] 10.222.0.15 () {60 vars in 1144 bytes} [Thu Sep 15 12:31:15 2022] POST /identity/v3/users => generated 298 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 12:31:15.609111 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10235997-74d0-45a3-88b2-bb2d520b7fd4 admin admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:31:15.619560 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10235997-74d0-45a3-88b2-bb2d520b7fd4 admin admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:31:15.624474 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb618c2-dc5a-4ea3-8d12-c44703decd32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:31:15.625090 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb618c2-dc5a-4ea3-8d12-c44703decd32 admin admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:31:15.625373 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb618c2-dc5a-4ea3-8d12-c44703decd32 admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:31:15.625646 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb618c2-dc5a-4ea3-8d12-c44703decd32 admin admin] PATH_INFO: `/v3/roles` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:31:15.626307 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb618c2-dc5a-4ea3-8d12-c44703decd32 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:31:15.628449 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb618c2-dc5a-4ea3-8d12-c44703decd32 admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:31:15.640399 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1367/2706] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 12:31:15 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 12:31:15.645256 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d5eeab-ef4f-4243-bc3b-34272e7b1af7 admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:31:15.655100 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d5eeab-ef4f-4243-bc3b-34272e7b1af7 admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:31:15.659628 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81b4019-7188-47e3-b8b3-05cc56b2cab4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:31:15.660272 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81b4019-7188-47e3-b8b3-05cc56b2cab4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:31:15.660550 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81b4019-7188-47e3-b8b3-05cc56b2cab4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:31:15.660844 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81b4019-7188-47e3-b8b3-05cc56b2cab4 admin admin] PATH_INFO: `/v3/projects/73f6f3290c3a4670b85a186afda49c02/users/a6e5b9c2c2c4455c8ff63113730f3a6f/roles/9c769b5c82884766b2c510b1f0f1364c` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:31:15.676911 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81b4019-7188-47e3-b8b3-05cc56b2cab4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=73f6f3290c3a4670b85a186afda49c02, user_id=a6e5b9c2c2c4455c8ff63113730f3a6f, role_id=9c769b5c82884766b2c510b1f0f1364c)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:31:15.679710 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81b4019-7188-47e3-b8b3-05cc56b2cab4 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:31:15.694402 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1340/2707] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 15 12:31:15 2022] PUT /identity/v3/projects/73f6f3290c3a4670b85a186afda49c02/users/a6e5b9c2c2c4455c8ff63113730f3a6f/roles/9c769b5c82884766b2c510b1f0f1364c => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:31:15.698394 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb618c2-dc5a-4ea3-8d12-c44703decd32 admin admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:31:15.708420 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb618c2-dc5a-4ea3-8d12-c44703decd32 admin admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:31:15.768544 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99ac469-3c7d-4397-b777-9b6acffb7cd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:31:15.769184 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99ac469-3c7d-4397-b777-9b6acffb7cd5 admin admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:31:15.769489 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99ac469-3c7d-4397-b777-9b6acffb7cd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:31:15.769761 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99ac469-3c7d-4397-b777-9b6acffb7cd5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:31:15.770396 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99ac469-3c7d-4397-b777-9b6acffb7cd5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:31:15.771971 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99ac469-3c7d-4397-b777-9b6acffb7cd5 admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:31:15.781615 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1368/2708] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 12:31:15 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 12:31:15.785810 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81b4019-7188-47e3-b8b3-05cc56b2cab4 admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:31:15.796721 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81b4019-7188-47e3-b8b3-05cc56b2cab4 admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:31:15.801868 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a45d175-9aca-48ef-9366-b6823a2e324b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:31:15.802535 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a45d175-9aca-48ef-9366-b6823a2e324b admin admin] REQUEST_METHOD: `PUT` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:31:15.802813 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a45d175-9aca-48ef-9366-b6823a2e324b admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:31:15.803090 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a45d175-9aca-48ef-9366-b6823a2e324b admin admin] PATH_INFO: `/v3/projects/73f6f3290c3a4670b85a186afda49c02/users/a6e5b9c2c2c4455c8ff63113730f3a6f/roles/9c769b5c82884766b2c510b1f0f1364c` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:31:15.805037 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a45d175-9aca-48ef-9366-b6823a2e324b admin admin] RBAC: Authorizing `identity:create_grant(project_id=73f6f3290c3a4670b85a186afda49c02, user_id=a6e5b9c2c2c4455c8ff63113730f3a6f, role_id=9c769b5c82884766b2c510b1f0f1364c)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:31:15.807643 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a45d175-9aca-48ef-9366-b6823a2e324b admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:31:15.818731 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1341/2709] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 15 12:31:15 2022] PUT /identity/v3/projects/73f6f3290c3a4670b85a186afda49c02/users/a6e5b9c2c2c4455c8ff63113730f3a6f/roles/9c769b5c82884766b2c510b1f0f1364c => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:31:15.829028 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b5d3ac-c4d6-49f7-affd-d9151109e237 None None] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:31:15.829317 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b5d3ac-c4d6-49f7-affd-d9151109e237 None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:31:15.829580 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b5d3ac-c4d6-49f7-affd-d9151109e237 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:31:15.860397 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.auth.core [None req-65b5d3ac-c4d6-49f7-affd-d9151109e237 None None] MFA Rules not processed for user `a6e5b9c2c2c4455c8ff63113730f3a6f`. Rule list: `[]` (Enabled: `True`). {{(pid=81449) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:31:15.861646 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-65b5d3ac-c4d6-49f7-affd-d9151109e237 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:31:15.911521 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:31:15.932510 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1369/2710] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 12:31:15 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:31:15.944336 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a45d175-9aca-48ef-9366-b6823a2e324b admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:31:16.004013 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a45d175-9aca-48ef-9366-b6823a2e324b admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:31:16.007726 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2458c05a-aa17-4d36-89cc-7bd0c3c4a2b3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8191d3ccb874529b32f8d0026db629d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '540a32ef88764e8c8d9201ae22f6ddaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:31:16.008355 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2458c05a-aa17-4d36-89cc-7bd0c3c4a2b3 service cinder] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:31:16.008605 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2458c05a-aa17-4d36-89cc-7bd0c3c4a2b3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:31:16.008909 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2458c05a-aa17-4d36-89cc-7bd0c3c4a2b3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:31:16.067867 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2458c05a-aa17-4d36-89cc-7bd0c3c4a2b3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:31:16.069331 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2458c05a-aa17-4d36-89cc-7bd0c3c4a2b3 service cinder] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:31:16.078695 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1342/2711] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 12:31:15 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 12:31:16.230831 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bc5abf-1a70-4cb7-b7bc-7c30e1952188 None None] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:31:16.231338 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bc5abf-1a70-4cb7-b7bc-7c30e1952188 None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:31:16.231431 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bc5abf-1a70-4cb7-b7bc-7c30e1952188 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:31:16.254917 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.auth.core [None req-72bc5abf-1a70-4cb7-b7bc-7c30e1952188 None None] MFA Rules not processed for user `527f4dd6031546c785bc6360dacb60af`. Rule list: `[]` (Enabled: `True`). {{(pid=81449) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:31:16.256368 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-72bc5abf-1a70-4cb7-b7bc-7c30e1952188 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:31:16.336530 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1370/2712] 10.222.0.15 () {58 vars in 967 bytes} [Thu Sep 15 12:31:16 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:31:16.353659 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2458c05a-aa17-4d36-89cc-7bd0c3c4a2b3 service cinder] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:31:16.355254 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-2458c05a-aa17-4d36-89cc-7bd0c3c4a2b3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:31:16.439057 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:31:16.480511 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2458c05a-aa17-4d36-89cc-7bd0c3c4a2b3 service cinder] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:31:16.480511 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf138341-cc2f-4c2b-a3de-92f3b0c19531 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa0d3dc5b7540d59fdd59bbf64a04b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '540a32ef88764e8c8d9201ae22f6ddaf', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:31:16.480511 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf138341-cc2f-4c2b-a3de-92f3b0c19531 service swift] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:31:16.480511 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf138341-cc2f-4c2b-a3de-92f3b0c19531 service swift] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:31:16.481037 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf138341-cc2f-4c2b-a3de-92f3b0c19531 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:31:16.491056 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf138341-cc2f-4c2b-a3de-92f3b0c19531 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:31:16.492664 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf138341-cc2f-4c2b-a3de-92f3b0c19531 service swift] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:31:16.503758 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1343/2713] 10.222.0.15 () {60 vars in 1384 bytes} [Thu Sep 15 12:31:16 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 15 12:31:17.698509 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72bc5abf-1a70-4cb7-b7bc-7c30e1952188 None None] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:31:17.714951 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72bc5abf-1a70-4cb7-b7bc-7c30e1952188 None None] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:31:17.718505 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd01426f-6334-46fc-b7ad-e7247f7d0cfd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8191d3ccb874529b32f8d0026db629d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '540a32ef88764e8c8d9201ae22f6ddaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:31:17.719129 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd01426f-6334-46fc-b7ad-e7247f7d0cfd service cinder] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:31:17.719403 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd01426f-6334-46fc-b7ad-e7247f7d0cfd service cinder] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:31:17.719666 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd01426f-6334-46fc-b7ad-e7247f7d0cfd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:31:17.779002 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd01426f-6334-46fc-b7ad-e7247f7d0cfd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:31:17.780428 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd01426f-6334-46fc-b7ad-e7247f7d0cfd service cinder] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:31:17.789950 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1371/2714] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 12:31:17 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 12:31:18.408735 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf138341-cc2f-4c2b-a3de-92f3b0c19531 service swift] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:31:18.419688 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf138341-cc2f-4c2b-a3de-92f3b0c19531 service swift] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:31:18.423692 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1437483c-274c-4f91-8dfd-0ddabcc1245c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8191d3ccb874529b32f8d0026db629d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '540a32ef88764e8c8d9201ae22f6ddaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:31:18.424300 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1437483c-274c-4f91-8dfd-0ddabcc1245c service cinder] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:31:18.424577 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1437483c-274c-4f91-8dfd-0ddabcc1245c service cinder] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:31:18.424874 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1437483c-274c-4f91-8dfd-0ddabcc1245c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:31:18.434105 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1437483c-274c-4f91-8dfd-0ddabcc1245c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:31:18.435502 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1437483c-274c-4f91-8dfd-0ddabcc1245c service cinder] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:31:18.445335 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1344/2715] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 12:31:18 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 12:31:18.590610 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd01426f-6334-46fc-b7ad-e7247f7d0cfd service cinder] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:31:18.592199 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-cd01426f-6334-46fc-b7ad-e7247f7d0cfd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:31:18.665456 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd01426f-6334-46fc-b7ad-e7247f7d0cfd service cinder] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:31:18.670049 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-765242cc-35a5-443d-97ee-c2959f6840ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:31:18.670676 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765242cc-35a5-443d-97ee-c2959f6840ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:31:18.670954 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765242cc-35a5-443d-97ee-c2959f6840ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:31:18.671225 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-765242cc-35a5-443d-97ee-c2959f6840ee admin admin] PATH_INFO: `/v3/users/0818d1847d5e42b6a942557f95b09d05` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:31:18.672362 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-765242cc-35a5-443d-97ee-c2959f6840ee admin admin] RBAC: Authorizing `identity:delete_user(user_id=0818d1847d5e42b6a942557f95b09d05)` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:31:18.674908 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-765242cc-35a5-443d-97ee-c2959f6840ee admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:31:18.716234 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:31:18.727519 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-765242cc-35a5-443d-97ee-c2959f6840ee admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0818d1847d5e42b6a942557f95b09d05'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:31:18.734921 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-765242cc-35a5-443d-97ee-c2959f6840ee admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0818d1847d5e42b6a942557f95b09d05'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:31:18.747411 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-765242cc-35a5-443d-97ee-c2959f6840ee admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0818d1847d5e42b6a942557f95b09d05'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:31:18.748084 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-765242cc-35a5-443d-97ee-c2959f6840ee admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0818d1847d5e42b6a942557f95b09d05'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:31:18.755236 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-765242cc-35a5-443d-97ee-c2959f6840ee admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0818d1847d5e42b6a942557f95b09d05'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:31:18.761287 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1372/2716] 10.222.0.15 () {58 vars in 1224 bytes} [Thu Sep 15 12:31:18 2022] DELETE /identity/v3/users/0818d1847d5e42b6a942557f95b09d05 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:31:18.941525 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1437483c-274c-4f91-8dfd-0ddabcc1245c service cinder] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:31:18.952461 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1437483c-274c-4f91-8dfd-0ddabcc1245c service cinder] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:31:19.032858 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0416d7ac-249c-4fd5-b6f8-d4526767e253 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:31:19.033836 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0416d7ac-249c-4fd5-b6f8-d4526767e253 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:31:19.034314 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0416d7ac-249c-4fd5-b6f8-d4526767e253 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:31:19.034854 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0416d7ac-249c-4fd5-b6f8-d4526767e253 admin admin] PATH_INFO: `/v3/projects/092e830b85d3479a952d2e4ada193ab3` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:31:19.038530 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0416d7ac-249c-4fd5-b6f8-d4526767e253 admin admin] RBAC: Authorizing `identity:delete_project(project_id=092e830b85d3479a952d2e4ada193ab3)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:31:19.041398 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0416d7ac-249c-4fd5-b6f8-d4526767e253 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:31:19.103784 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-0416d7ac-249c-4fd5-b6f8-d4526767e253 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '092e830b85d3479a952d2e4ada193ab3', 'request_id': 'req-0416d7ac-249c-4fd5-b6f8-d4526767e253'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:31:19.111527 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-0416d7ac-249c-4fd5-b6f8-d4526767e253 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '092e830b85d3479a952d2e4ada193ab3', 'request_id': 'req-0416d7ac-249c-4fd5-b6f8-d4526767e253'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:31:19.122385 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-0416d7ac-249c-4fd5-b6f8-d4526767e253 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '092e830b85d3479a952d2e4ada193ab3', 'request_id': 'req-0416d7ac-249c-4fd5-b6f8-d4526767e253'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:31:19.134813 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-0416d7ac-249c-4fd5-b6f8-d4526767e253 admin admin] The token cache is being invalidate because project 092e830b85d3479a952d2e4ada193ab3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81448) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 12:31:19.135106 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-0416d7ac-249c-4fd5-b6f8-d4526767e253 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:31:19.135889 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-0416d7ac-249c-4fd5-b6f8-d4526767e253 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:31:19.137708 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1345/2717] 10.222.0.15 () {58 vars in 1233 bytes} [Thu Sep 15 12:31:18 2022] DELETE /identity/v3/projects/092e830b85d3479a952d2e4ada193ab3 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:31:19.142745 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e067ad-b270-4e4e-b011-f093ae1d451c None None] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:31:19.143110 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e067ad-b270-4e4e-b011-f093ae1d451c None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:31:19.143389 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e067ad-b270-4e4e-b011-f093ae1d451c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:31:19.164142 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.auth.core [None req-e1e067ad-b270-4e4e-b011-f093ae1d451c None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81449) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:31:19.165322 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-e1e067ad-b270-4e4e-b011-f093ae1d451c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:31:19.254523 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1373/2718] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 12:31:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:31:19.255794 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0165b160-7da8-48b7-b17d-328dd885d1e0 None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:31:19.256127 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0165b160-7da8-48b7-b17d-328dd885d1e0 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:31:19.256431 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0165b160-7da8-48b7-b17d-328dd885d1e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:31:19.288698 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-0165b160-7da8-48b7-b17d-328dd885d1e0 None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:31:19.290054 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-0165b160-7da8-48b7-b17d-328dd885d1e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:31:19.301215 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1346/2719] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 12:31:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 12:31:19.305368 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e067ad-b270-4e4e-b011-f093ae1d451c None None] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:31:19.315181 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e067ad-b270-4e4e-b011-f093ae1d451c None None] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:31:19.319799 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a47e30f-f405-455c-84e1-0147b23727d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:31:19.320346 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a47e30f-f405-455c-84e1-0147b23727d5 admin admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:31:19.320625 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a47e30f-f405-455c-84e1-0147b23727d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:31:19.320928 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a47e30f-f405-455c-84e1-0147b23727d5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:31:19.321572 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a47e30f-f405-455c-84e1-0147b23727d5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81449) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:31:19.321874 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a47e30f-f405-455c-84e1-0147b23727d5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:31:19.323456 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a47e30f-f405-455c-84e1-0147b23727d5 admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:31:19.335561 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1374/2720] 10.222.0.15 () {58 vars in 1153 bytes} [Thu Sep 15 12:31:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:31:19.340127 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0165b160-7da8-48b7-b17d-328dd885d1e0 None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:31:19.351386 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0165b160-7da8-48b7-b17d-328dd885d1e0 None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:31:19.356091 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5d9967-ac53-4667-8319-80f1d0951572 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:31:19.356696 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5d9967-ac53-4667-8319-80f1d0951572 admin admin] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:31:19.357006 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5d9967-ac53-4667-8319-80f1d0951572 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:31:19.357272 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5d9967-ac53-4667-8319-80f1d0951572 admin admin] PATH_INFO: `/v3/projects` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:31:19.357908 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5d9967-ac53-4667-8319-80f1d0951572 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:31:19.359792 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5d9967-ac53-4667-8319-80f1d0951572 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:31:19.386108 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1347/2721] 10.222.0.15 () {60 vars in 1153 bytes} [Thu Sep 15 12:31:19 2022] POST /identity/v3/projects => generated 371 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 12:31:19.390402 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a47e30f-f405-455c-84e1-0147b23727d5 admin admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:31:19.400347 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a47e30f-f405-455c-84e1-0147b23727d5 admin admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:31:19.454880 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb743f4-3c3b-4cb3-9c9c-e46767ab9231 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:31:19.456012 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb743f4-3c3b-4cb3-9c9c-e46767ab9231 admin admin] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:31:19.456289 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb743f4-3c3b-4cb3-9c9c-e46767ab9231 admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:31:19.456566 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb743f4-3c3b-4cb3-9c9c-e46767ab9231 admin admin] PATH_INFO: `/v3/users` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:31:19.457209 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb743f4-3c3b-4cb3-9c9c-e46767ab9231 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:31:19.459102 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb743f4-3c3b-4cb3-9c9c-e46767ab9231 admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:31:19.480212 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1375/2722] 10.222.0.15 () {60 vars in 1144 bytes} [Thu Sep 15 12:31:19 2022] POST /identity/v3/users => generated 286 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 12:31:19.485488 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5d9967-ac53-4667-8319-80f1d0951572 admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:31:19.497504 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5d9967-ac53-4667-8319-80f1d0951572 admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:31:19.502105 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45723c23-9cb5-4fe7-8088-0eb3f3225e82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:31:19.502763 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45723c23-9cb5-4fe7-8088-0eb3f3225e82 admin admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:31:19.503037 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45723c23-9cb5-4fe7-8088-0eb3f3225e82 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:31:19.503305 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45723c23-9cb5-4fe7-8088-0eb3f3225e82 admin admin] PATH_INFO: `/v3/roles` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:31:19.504612 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45723c23-9cb5-4fe7-8088-0eb3f3225e82 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:31:19.506442 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45723c23-9cb5-4fe7-8088-0eb3f3225e82 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:31:19.521334 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1348/2723] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 12:31:19 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 12:31:19.523800 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb743f4-3c3b-4cb3-9c9c-e46767ab9231 admin admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:31:19.535604 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb743f4-3c3b-4cb3-9c9c-e46767ab9231 admin admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:31:19.540457 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9658e0f4-f69c-44c3-8727-0ef51ae09048 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:31:19.541106 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9658e0f4-f69c-44c3-8727-0ef51ae09048 admin admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:31:19.541387 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9658e0f4-f69c-44c3-8727-0ef51ae09048 admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:31:19.541779 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9658e0f4-f69c-44c3-8727-0ef51ae09048 admin admin] PATH_INFO: `/v3/roles` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:31:19.542422 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9658e0f4-f69c-44c3-8727-0ef51ae09048 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:31:19.544068 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9658e0f4-f69c-44c3-8727-0ef51ae09048 admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:31:19.556739 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1376/2724] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 12:31:19 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 12:31:19.560997 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45723c23-9cb5-4fe7-8088-0eb3f3225e82 admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:31:19.572464 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45723c23-9cb5-4fe7-8088-0eb3f3225e82 admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:31:19.577318 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae660066-ae0e-494d-9cfa-43fee064fd49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:31:19.577980 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae660066-ae0e-494d-9cfa-43fee064fd49 admin admin] REQUEST_METHOD: `PUT` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:31:19.578254 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae660066-ae0e-494d-9cfa-43fee064fd49 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:31:19.578522 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae660066-ae0e-494d-9cfa-43fee064fd49 admin admin] PATH_INFO: `/v3/projects/c11c9b11a9ce4e78be9d9029fd9bcfdd/users/bcd4003f953f4f1999e6e6de41d82859/roles/0679d6628a25438d90dc4bdc0401eda2` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:31:19.598567 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae660066-ae0e-494d-9cfa-43fee064fd49 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c11c9b11a9ce4e78be9d9029fd9bcfdd, user_id=bcd4003f953f4f1999e6e6de41d82859, role_id=0679d6628a25438d90dc4bdc0401eda2)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:31:19.601687 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae660066-ae0e-494d-9cfa-43fee064fd49 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:31:19.604116 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:31:19.617843 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1349/2725] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 15 12:31:19 2022] PUT /identity/v3/projects/c11c9b11a9ce4e78be9d9029fd9bcfdd/users/bcd4003f953f4f1999e6e6de41d82859/roles/0679d6628a25438d90dc4bdc0401eda2 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 12:31:19.628005 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9f8304-54db-43c5-99b5-bb4ef55c9624 None None] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:31:19.628441 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9f8304-54db-43c5-99b5-bb4ef55c9624 None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:31:19.628854 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9f8304-54db-43c5-99b5-bb4ef55c9624 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:31:19.657527 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.auth.core [None req-2b9f8304-54db-43c5-99b5-bb4ef55c9624 None None] MFA Rules not processed for user `bcd4003f953f4f1999e6e6de41d82859`. Rule list: `[]` (Enabled: `True`). {{(pid=81449) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:31:19.659433 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-2b9f8304-54db-43c5-99b5-bb4ef55c9624 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:31:19.665963 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:31:19.740432 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1377/2726] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 12:31:19 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:31:19.751722 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae660066-ae0e-494d-9cfa-43fee064fd49 admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:31:19.753691 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-ae660066-ae0e-494d-9cfa-43fee064fd49 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:31:19.841288 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae660066-ae0e-494d-9cfa-43fee064fd49 admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:31:19.845306 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e44ea52-5282-460e-ac74-56c0bf018bfe service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8191d3ccb874529b32f8d0026db629d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '540a32ef88764e8c8d9201ae22f6ddaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:31:19.846043 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e44ea52-5282-460e-ac74-56c0bf018bfe service cinder] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:31:19.846344 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e44ea52-5282-460e-ac74-56c0bf018bfe service cinder] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:31:19.846645 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e44ea52-5282-460e-ac74-56c0bf018bfe service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:31:19.858072 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e44ea52-5282-460e-ac74-56c0bf018bfe service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:31:19.860471 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e44ea52-5282-460e-ac74-56c0bf018bfe service cinder] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:31:19.873687 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1350/2727] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 12:31:19 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 12:31:19.906162 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9f8304-54db-43c5-99b5-bb4ef55c9624 None None] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:31:19.979042 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9f8304-54db-43c5-99b5-bb4ef55c9624 None None] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:31:19.983659 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040864e5-9af2-4115-ab3f-f99d582aa19c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:31:19.984294 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040864e5-9af2-4115-ab3f-f99d582aa19c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:31:19.984576 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040864e5-9af2-4115-ab3f-f99d582aa19c admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:31:19.984878 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040864e5-9af2-4115-ab3f-f99d582aa19c admin admin] PATH_INFO: `/v3/users/bcd4003f953f4f1999e6e6de41d82859` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:31:19.986018 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040864e5-9af2-4115-ab3f-f99d582aa19c admin admin] RBAC: Authorizing `identity:delete_user(user_id=bcd4003f953f4f1999e6e6de41d82859)` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:31:19.988518 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040864e5-9af2-4115-ab3f-f99d582aa19c admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:31:20.034317 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-040864e5-9af2-4115-ab3f-f99d582aa19c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bcd4003f953f4f1999e6e6de41d82859'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:31:20.042193 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-040864e5-9af2-4115-ab3f-f99d582aa19c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bcd4003f953f4f1999e6e6de41d82859'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:31:20.054744 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-040864e5-9af2-4115-ab3f-f99d582aa19c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bcd4003f953f4f1999e6e6de41d82859'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:31:20.055371 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-040864e5-9af2-4115-ab3f-f99d582aa19c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bcd4003f953f4f1999e6e6de41d82859'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:31:20.062155 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-040864e5-9af2-4115-ab3f-f99d582aa19c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bcd4003f953f4f1999e6e6de41d82859'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:31:20.068558 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1378/2728] 10.222.0.15 () {58 vars in 1224 bytes} [Thu Sep 15 12:31:19 2022] DELETE /identity/v3/users/bcd4003f953f4f1999e6e6de41d82859 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:31:20.073881 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df159d45-5887-45ae-8757-c8b6ca92a023 None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:31:20.074170 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df159d45-5887-45ae-8757-c8b6ca92a023 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:31:20.074439 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df159d45-5887-45ae-8757-c8b6ca92a023 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:31:20.093983 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-df159d45-5887-45ae-8757-c8b6ca92a023 None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:31:20.095222 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-df159d45-5887-45ae-8757-c8b6ca92a023 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:31:20.155936 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1351/2729] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 12:31:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:31:20.167017 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040864e5-9af2-4115-ab3f-f99d582aa19c admin admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:31:20.168716 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-040864e5-9af2-4115-ab3f-f99d582aa19c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:31:20.202241 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:31:20.232244 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040864e5-9af2-4115-ab3f-f99d582aa19c admin admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:31:20.236167 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf91d23-bfe1-45dc-98a9-d745a2ead9df service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfbb4d6809064f9cb9e06626ed7d6814', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '540a32ef88764e8c8d9201ae22f6ddaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:31:20.236839 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf91d23-bfe1-45dc-98a9-d745a2ead9df service neutron] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:31:20.237121 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf91d23-bfe1-45dc-98a9-d745a2ead9df service neutron] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:31:20.237389 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf91d23-bfe1-45dc-98a9-d745a2ead9df service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:31:20.248831 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf91d23-bfe1-45dc-98a9-d745a2ead9df service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:31:20.250399 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf91d23-bfe1-45dc-98a9-d745a2ead9df service neutron] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:31:20.260497 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1379/2730] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 12:31:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 15 12:31:20.564383 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df159d45-5887-45ae-8757-c8b6ca92a023 None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:31:20.566214 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-df159d45-5887-45ae-8757-c8b6ca92a023 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:31:20.587461 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df159d45-5887-45ae-8757-c8b6ca92a023 None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:31:20.594271 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff99da0-56c1-49f6-8b58-8cd7ef747a8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:31:20.594858 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff99da0-56c1-49f6-8b58-8cd7ef747a8c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:31:20.595140 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff99da0-56c1-49f6-8b58-8cd7ef747a8c admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:31:20.595437 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff99da0-56c1-49f6-8b58-8cd7ef747a8c admin admin] PATH_INFO: `/v3/projects/c11c9b11a9ce4e78be9d9029fd9bcfdd` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:31:20.597340 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff99da0-56c1-49f6-8b58-8cd7ef747a8c admin admin] RBAC: Authorizing `identity:delete_project(project_id=c11c9b11a9ce4e78be9d9029fd9bcfdd)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:31:20.599198 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff99da0-56c1-49f6-8b58-8cd7ef747a8c admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:31:20.646612 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-0ff99da0-56c1-49f6-8b58-8cd7ef747a8c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c11c9b11a9ce4e78be9d9029fd9bcfdd', 'request_id': 'req-0ff99da0-56c1-49f6-8b58-8cd7ef747a8c'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:31:20.654258 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-0ff99da0-56c1-49f6-8b58-8cd7ef747a8c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c11c9b11a9ce4e78be9d9029fd9bcfdd', 'request_id': 'req-0ff99da0-56c1-49f6-8b58-8cd7ef747a8c'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:31:20.660614 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-0ff99da0-56c1-49f6-8b58-8cd7ef747a8c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c11c9b11a9ce4e78be9d9029fd9bcfdd', 'request_id': 'req-0ff99da0-56c1-49f6-8b58-8cd7ef747a8c'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:31:20.676148 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-0ff99da0-56c1-49f6-8b58-8cd7ef747a8c admin admin] The token cache is being invalidate because project c11c9b11a9ce4e78be9d9029fd9bcfdd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81448) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 12:31:20.676435 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-0ff99da0-56c1-49f6-8b58-8cd7ef747a8c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:31:20.677119 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-0ff99da0-56c1-49f6-8b58-8cd7ef747a8c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:31:20.678595 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1352/2731] 10.222.0.15 () {58 vars in 1233 bytes} [Thu Sep 15 12:31:20 2022] DELETE /identity/v3/projects/c11c9b11a9ce4e78be9d9029fd9bcfdd => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:31:20.684261 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598f4224-d54a-416c-829e-0a4519b1b1fd None None] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:31:20.684686 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598f4224-d54a-416c-829e-0a4519b1b1fd None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:31:20.685286 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598f4224-d54a-416c-829e-0a4519b1b1fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:31:20.709061 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.auth.core [None req-598f4224-d54a-416c-829e-0a4519b1b1fd None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81449) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:31:20.710551 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-598f4224-d54a-416c-829e-0a4519b1b1fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:31:20.780498 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1380/2732] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 12:31:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:31:20.788955 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac5f895-0dad-447c-a864-642cae7f4c25 None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:31:20.789260 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac5f895-0dad-447c-a864-642cae7f4c25 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:31:20.789542 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac5f895-0dad-447c-a864-642cae7f4c25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:31:20.813671 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-aac5f895-0dad-447c-a864-642cae7f4c25 None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:31:20.815427 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-aac5f895-0dad-447c-a864-642cae7f4c25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:31:20.824241 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1353/2733] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 12:31:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:31:20.828596 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598f4224-d54a-416c-829e-0a4519b1b1fd None None] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:31:20.839331 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598f4224-d54a-416c-829e-0a4519b1b1fd None None] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:31:20.844817 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8d4709-4b41-4eff-a5aa-ad0b721ad555 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:31:20.845654 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8d4709-4b41-4eff-a5aa-ad0b721ad555 admin admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:31:20.846083 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8d4709-4b41-4eff-a5aa-ad0b721ad555 admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:31:20.846455 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8d4709-4b41-4eff-a5aa-ad0b721ad555 admin admin] PATH_INFO: `/v3/domains` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:31:20.847350 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f8d4709-4b41-4eff-a5aa-ad0b721ad555 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81449) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:31:20.847791 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f8d4709-4b41-4eff-a5aa-ad0b721ad555 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:31:20.849806 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f8d4709-4b41-4eff-a5aa-ad0b721ad555 admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:31:20.862391 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1381/2734] 10.222.0.15 () {58 vars in 1153 bytes} [Thu Sep 15 12:31:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:31:20.866692 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac5f895-0dad-447c-a864-642cae7f4c25 None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:31:20.876005 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac5f895-0dad-447c-a864-642cae7f4c25 None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:31:20.881234 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34168262-b401-4fc8-be11-1c6e7bcd4551 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:31:20.881839 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34168262-b401-4fc8-be11-1c6e7bcd4551 admin admin] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:31:20.882118 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34168262-b401-4fc8-be11-1c6e7bcd4551 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:31:20.882386 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34168262-b401-4fc8-be11-1c6e7bcd4551 admin admin] PATH_INFO: `/v3/projects` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:31:20.883025 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34168262-b401-4fc8-be11-1c6e7bcd4551 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:31:20.884935 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34168262-b401-4fc8-be11-1c6e7bcd4551 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:31:20.897777 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:INSERT=2,keystone:SELECT=91,keystone:DELETE=7 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:31:20.918305 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1354/2735] 10.222.0.15 () {60 vars in 1153 bytes} [Thu Sep 15 12:31:20 2022] POST /identity/v3/projects => generated 375 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 12:31:20.922749 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8d4709-4b41-4eff-a5aa-ad0b721ad555 admin admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:31:20.935147 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8d4709-4b41-4eff-a5aa-ad0b721ad555 admin admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:31:21.003655 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd215eec-293f-423a-9157-275ee2edf9d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:31:21.004405 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd215eec-293f-423a-9157-275ee2edf9d4 admin admin] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:31:21.004707 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd215eec-293f-423a-9157-275ee2edf9d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:31:21.005091 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd215eec-293f-423a-9157-275ee2edf9d4 admin admin] PATH_INFO: `/v3/users` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:31:21.005740 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd215eec-293f-423a-9157-275ee2edf9d4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:31:21.009125 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd215eec-293f-423a-9157-275ee2edf9d4 admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:31:21.033620 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1382/2736] 10.222.0.15 () {60 vars in 1144 bytes} [Thu Sep 15 12:31:20 2022] POST /identity/v3/users => generated 288 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 12:31:21.037863 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34168262-b401-4fc8-be11-1c6e7bcd4551 admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:31:21.057208 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34168262-b401-4fc8-be11-1c6e7bcd4551 admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:31:21.057208 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59795a11-0853-4833-83e2-08f98d103476 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:31:21.057208 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59795a11-0853-4833-83e2-08f98d103476 admin admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:31:21.057208 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59795a11-0853-4833-83e2-08f98d103476 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:31:21.057651 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59795a11-0853-4833-83e2-08f98d103476 admin admin] PATH_INFO: `/v3/roles` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:31:21.057651 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59795a11-0853-4833-83e2-08f98d103476 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:31:21.057651 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59795a11-0853-4833-83e2-08f98d103476 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:31:21.068977 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1355/2737] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 12:31:21 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 12:31:21.074674 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd215eec-293f-423a-9157-275ee2edf9d4 admin admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:31:21.085750 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd215eec-293f-423a-9157-275ee2edf9d4 admin admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:31:21.092904 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb1e912-48ef-4a20-876e-3151470ffb80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:31:21.093534 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb1e912-48ef-4a20-876e-3151470ffb80 admin admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:31:21.093809 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb1e912-48ef-4a20-876e-3151470ffb80 admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:31:21.094077 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb1e912-48ef-4a20-876e-3151470ffb80 admin admin] PATH_INFO: `/v3/roles` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:31:21.094726 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb1e912-48ef-4a20-876e-3151470ffb80 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:31:21.096326 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb1e912-48ef-4a20-876e-3151470ffb80 admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:31:21.110266 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1383/2738] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 12:31:21 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 12:31:21.114770 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59795a11-0853-4833-83e2-08f98d103476 admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:31:21.124774 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59795a11-0853-4833-83e2-08f98d103476 admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:31:21.131726 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49dfc3f-879f-471a-8081-a0b08ded2c6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:31:21.132321 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49dfc3f-879f-471a-8081-a0b08ded2c6f admin admin] REQUEST_METHOD: `PUT` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:31:21.132606 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49dfc3f-879f-471a-8081-a0b08ded2c6f admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:31:21.135482 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49dfc3f-879f-471a-8081-a0b08ded2c6f admin admin] PATH_INFO: `/v3/projects/617b2e1cc38d4faabe0ac41504480cb4/users/b87556e083ae4854a4d26336490ccd1d/roles/0679d6628a25438d90dc4bdc0401eda2` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:31:21.154046 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49dfc3f-879f-471a-8081-a0b08ded2c6f admin admin] RBAC: Authorizing `identity:create_grant(project_id=617b2e1cc38d4faabe0ac41504480cb4, user_id=b87556e083ae4854a4d26336490ccd1d, role_id=0679d6628a25438d90dc4bdc0401eda2)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:31:21.157172 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49dfc3f-879f-471a-8081-a0b08ded2c6f admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:31:21.180219 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1356/2739] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 15 12:31:21 2022] PUT /identity/v3/projects/617b2e1cc38d4faabe0ac41504480cb4/users/b87556e083ae4854a4d26336490ccd1d/roles/0679d6628a25438d90dc4bdc0401eda2 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:31:21.185984 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a1fe8e-f731-4822-b656-15a1c1e19ac3 None None] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:31:21.186272 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a1fe8e-f731-4822-b656-15a1c1e19ac3 None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:31:21.186555 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a1fe8e-f731-4822-b656-15a1c1e19ac3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:31:21.219764 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.auth.core [None req-43a1fe8e-f731-4822-b656-15a1c1e19ac3 None None] MFA Rules not processed for user `b87556e083ae4854a4d26336490ccd1d`. Rule list: `[]` (Enabled: `True`). {{(pid=81449) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:31:21.221623 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-43a1fe8e-f731-4822-b656-15a1c1e19ac3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:31:21.249143 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:31:21.298061 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1384/2740] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 12:31:21 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:31:21.311883 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49dfc3f-879f-471a-8081-a0b08ded2c6f admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:31:21.313967 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-b49dfc3f-879f-471a-8081-a0b08ded2c6f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:31:21.380450 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49dfc3f-879f-471a-8081-a0b08ded2c6f admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:31:21.384153 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66543ad-d2a6-4ca3-9417-504369eb3ea5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8191d3ccb874529b32f8d0026db629d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '540a32ef88764e8c8d9201ae22f6ddaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:31:21.384778 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66543ad-d2a6-4ca3-9417-504369eb3ea5 service cinder] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:31:21.385061 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66543ad-d2a6-4ca3-9417-504369eb3ea5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:31:21.385328 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66543ad-d2a6-4ca3-9417-504369eb3ea5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:31:21.396101 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66543ad-d2a6-4ca3-9417-504369eb3ea5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:31:21.397602 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66543ad-d2a6-4ca3-9417-504369eb3ea5 service cinder] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:31:21.406865 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1357/2741] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 12:31:21 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 12:31:22.415598 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a1fe8e-f731-4822-b656-15a1c1e19ac3 None None] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:31:22.417053 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-43a1fe8e-f731-4822-b656-15a1c1e19ac3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:31:22.482589 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a1fe8e-f731-4822-b656-15a1c1e19ac3 None None] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:31:22.487445 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d34262-8704-4a49-bf5f-63a4b2620ef1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:31:22.488071 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d34262-8704-4a49-bf5f-63a4b2620ef1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:31:22.488348 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d34262-8704-4a49-bf5f-63a4b2620ef1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:31:22.488617 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d34262-8704-4a49-bf5f-63a4b2620ef1 admin admin] PATH_INFO: `/v3/users/816b6ea10d8140d8be8de34b4a312224` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:31:22.489757 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d34262-8704-4a49-bf5f-63a4b2620ef1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=816b6ea10d8140d8be8de34b4a312224)` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:31:22.491670 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d34262-8704-4a49-bf5f-63a4b2620ef1 admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:31:22.537995 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-c2d34262-8704-4a49-bf5f-63a4b2620ef1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '816b6ea10d8140d8be8de34b4a312224'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:31:22.548312 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-c2d34262-8704-4a49-bf5f-63a4b2620ef1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '816b6ea10d8140d8be8de34b4a312224'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:31:22.561427 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-c2d34262-8704-4a49-bf5f-63a4b2620ef1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '816b6ea10d8140d8be8de34b4a312224'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:31:22.562109 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-c2d34262-8704-4a49-bf5f-63a4b2620ef1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '816b6ea10d8140d8be8de34b4a312224'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:31:22.569095 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-c2d34262-8704-4a49-bf5f-63a4b2620ef1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '816b6ea10d8140d8be8de34b4a312224'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:31:22.574897 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1385/2742] 10.222.0.15 () {58 vars in 1224 bytes} [Thu Sep 15 12:31:22 2022] DELETE /identity/v3/users/816b6ea10d8140d8be8de34b4a312224 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:31:22.579384 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad773eb3-8702-4029-b25f-940319c480a2 None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:31:22.579856 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad773eb3-8702-4029-b25f-940319c480a2 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:31:22.579981 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad773eb3-8702-4029-b25f-940319c480a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:31:22.601341 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-ad773eb3-8702-4029-b25f-940319c480a2 None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:31:22.602584 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-ad773eb3-8702-4029-b25f-940319c480a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:31:22.661234 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1358/2743] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 12:31:22 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:31:22.671049 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d34262-8704-4a49-bf5f-63a4b2620ef1 admin admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:31:22.672632 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-c2d34262-8704-4a49-bf5f-63a4b2620ef1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:31:22.721835 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:31:22.736226 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d34262-8704-4a49-bf5f-63a4b2620ef1 admin admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:31:22.739790 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c10e895-cb8f-482a-b651-f06a1f2519f2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfbb4d6809064f9cb9e06626ed7d6814', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '540a32ef88764e8c8d9201ae22f6ddaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:31:22.740392 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c10e895-cb8f-482a-b651-f06a1f2519f2 service neutron] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:31:22.740664 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c10e895-cb8f-482a-b651-f06a1f2519f2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:31:22.740962 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c10e895-cb8f-482a-b651-f06a1f2519f2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:31:22.750302 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c10e895-cb8f-482a-b651-f06a1f2519f2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:31:22.751706 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c10e895-cb8f-482a-b651-f06a1f2519f2 service neutron] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:31:22.764105 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1386/2744] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 12:31:22 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 15 12:31:22.780026 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064d423f-aae9-44f7-b599-059344b1837a None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:31:22.780307 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064d423f-aae9-44f7-b599-059344b1837a None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:31:22.780577 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064d423f-aae9-44f7-b599-059344b1837a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:31:22.800066 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-064d423f-aae9-44f7-b599-059344b1837a None None] MFA Rules not processed for user `b87556e083ae4854a4d26336490ccd1d`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:31:22.801308 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-064d423f-aae9-44f7-b599-059344b1837a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:31:22.865500 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1359/2745] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 12:31:22 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 12:31:23.279640 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c10e895-cb8f-482a-b651-f06a1f2519f2 service neutron] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:31:23.281306 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-3c10e895-cb8f-482a-b651-f06a1f2519f2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:31:23.299385 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c10e895-cb8f-482a-b651-f06a1f2519f2 service neutron] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:31:23.304136 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e95cfb-456e-4700-a666-511b25f680e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:31:23.304955 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e95cfb-456e-4700-a666-511b25f680e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:31:23.305446 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e95cfb-456e-4700-a666-511b25f680e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:31:23.305989 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e95cfb-456e-4700-a666-511b25f680e6 admin admin] PATH_INFO: `/v3/projects/9f94d0d282214ac8bc1f50d83f14f517` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:31:23.307220 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e95cfb-456e-4700-a666-511b25f680e6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9f94d0d282214ac8bc1f50d83f14f517)` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:31:23.309508 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e95cfb-456e-4700-a666-511b25f680e6 admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:31:23.371525 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-e4e95cfb-456e-4700-a666-511b25f680e6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9f94d0d282214ac8bc1f50d83f14f517', 'request_id': 'req-e4e95cfb-456e-4700-a666-511b25f680e6'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:31:23.379661 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-e4e95cfb-456e-4700-a666-511b25f680e6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9f94d0d282214ac8bc1f50d83f14f517', 'request_id': 'req-e4e95cfb-456e-4700-a666-511b25f680e6'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:31:23.386154 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-e4e95cfb-456e-4700-a666-511b25f680e6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9f94d0d282214ac8bc1f50d83f14f517', 'request_id': 'req-e4e95cfb-456e-4700-a666-511b25f680e6'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:31:23.406367 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-e4e95cfb-456e-4700-a666-511b25f680e6 admin admin] The token cache is being invalidate because project 9f94d0d282214ac8bc1f50d83f14f517 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81449) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 12:31:23.406697 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-e4e95cfb-456e-4700-a666-511b25f680e6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:31:23.407335 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-e4e95cfb-456e-4700-a666-511b25f680e6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:31:23.412073 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1387/2746] 10.222.0.15 () {58 vars in 1233 bytes} [Thu Sep 15 12:31:23 2022] DELETE /identity/v3/projects/9f94d0d282214ac8bc1f50d83f14f517 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:31:23.413318 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064d423f-aae9-44f7-b599-059344b1837a None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:31:23.420606 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:31:23.426003 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064d423f-aae9-44f7-b599-059344b1837a None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:31:23.501537 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af533c2-81de-4d61-8bf5-85eedef789b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:31:23.502304 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af533c2-81de-4d61-8bf5-85eedef789b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:31:23.502581 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af533c2-81de-4d61-8bf5-85eedef789b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:31:23.502848 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af533c2-81de-4d61-8bf5-85eedef789b0 admin admin] PATH_INFO: `/v3/users/f9f10b0cb37d4eb5acf3e84df8e196a6` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:31:23.504011 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af533c2-81de-4d61-8bf5-85eedef789b0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f9f10b0cb37d4eb5acf3e84df8e196a6)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:31:23.506626 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af533c2-81de-4d61-8bf5-85eedef789b0 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:31:23.566895 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-8af533c2-81de-4d61-8bf5-85eedef789b0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f9f10b0cb37d4eb5acf3e84df8e196a6'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:31:23.579859 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-8af533c2-81de-4d61-8bf5-85eedef789b0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f9f10b0cb37d4eb5acf3e84df8e196a6'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:31:23.588561 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-8af533c2-81de-4d61-8bf5-85eedef789b0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f9f10b0cb37d4eb5acf3e84df8e196a6'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:31:23.589429 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-8af533c2-81de-4d61-8bf5-85eedef789b0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f9f10b0cb37d4eb5acf3e84df8e196a6'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:31:23.600290 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-8af533c2-81de-4d61-8bf5-85eedef789b0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f9f10b0cb37d4eb5acf3e84df8e196a6'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:31:23.606773 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1360/2747] 10.222.0.15 () {58 vars in 1224 bytes} [Thu Sep 15 12:31:23 2022] DELETE /identity/v3/users/f9f10b0cb37d4eb5acf3e84df8e196a6 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 12:31:23.910929 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e95cfb-456e-4700-a666-511b25f680e6 admin admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:31:23.917146 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-e4e95cfb-456e-4700-a666-511b25f680e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:31:23.937714 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e95cfb-456e-4700-a666-511b25f680e6 admin admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:31:24.011724 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e100bd-cd2f-4061-8cd2-63641ed9cee1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:31:24.012361 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e100bd-cd2f-4061-8cd2-63641ed9cee1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:31:24.012639 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e100bd-cd2f-4061-8cd2-63641ed9cee1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:31:24.012953 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e100bd-cd2f-4061-8cd2-63641ed9cee1 admin admin] PATH_INFO: `/v3/projects/b1d8e76a968b42cfa496700755b49a4f` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:31:24.014079 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e100bd-cd2f-4061-8cd2-63641ed9cee1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b1d8e76a968b42cfa496700755b49a4f)` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:31:24.016009 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e100bd-cd2f-4061-8cd2-63641ed9cee1 admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:31:24.066507 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-d9e100bd-cd2f-4061-8cd2-63641ed9cee1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b1d8e76a968b42cfa496700755b49a4f', 'request_id': 'req-d9e100bd-cd2f-4061-8cd2-63641ed9cee1'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:31:24.074920 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-d9e100bd-cd2f-4061-8cd2-63641ed9cee1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b1d8e76a968b42cfa496700755b49a4f', 'request_id': 'req-d9e100bd-cd2f-4061-8cd2-63641ed9cee1'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:31:24.080868 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-d9e100bd-cd2f-4061-8cd2-63641ed9cee1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b1d8e76a968b42cfa496700755b49a4f', 'request_id': 'req-d9e100bd-cd2f-4061-8cd2-63641ed9cee1'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:31:24.089264 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:31:24.093859 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-d9e100bd-cd2f-4061-8cd2-63641ed9cee1 admin admin] The token cache is being invalidate because project b1d8e76a968b42cfa496700755b49a4f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81449) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 12:31:24.094226 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-d9e100bd-cd2f-4061-8cd2-63641ed9cee1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:31:24.094921 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-d9e100bd-cd2f-4061-8cd2-63641ed9cee1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:31:24.096665 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1388/2748] 10.222.0.15 () {58 vars in 1233 bytes} [Thu Sep 15 12:31:23 2022] DELETE /identity/v3/projects/b1d8e76a968b42cfa496700755b49a4f => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:31:24.100917 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af533c2-81de-4d61-8bf5-85eedef789b0 admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:31:24.102774 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-8af533c2-81de-4d61-8bf5-85eedef789b0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:31:24.115919 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af533c2-81de-4d61-8bf5-85eedef789b0 admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:31:24.176187 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8337eb-4d41-464a-b604-1c44b53ddcc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:31:24.176930 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8337eb-4d41-464a-b604-1c44b53ddcc9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:31:24.177228 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8337eb-4d41-464a-b604-1c44b53ddcc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:31:24.177505 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8337eb-4d41-464a-b604-1c44b53ddcc9 admin admin] PATH_INFO: `/v3/users/a6e5b9c2c2c4455c8ff63113730f3a6f` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:31:24.178659 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8337eb-4d41-464a-b604-1c44b53ddcc9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a6e5b9c2c2c4455c8ff63113730f3a6f)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:31:24.180663 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8337eb-4d41-464a-b604-1c44b53ddcc9 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:31:24.210237 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:31:24.230076 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-fd8337eb-4d41-464a-b604-1c44b53ddcc9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a6e5b9c2c2c4455c8ff63113730f3a6f'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:31:24.243120 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-fd8337eb-4d41-464a-b604-1c44b53ddcc9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a6e5b9c2c2c4455c8ff63113730f3a6f'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:31:24.251749 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-fd8337eb-4d41-464a-b604-1c44b53ddcc9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a6e5b9c2c2c4455c8ff63113730f3a6f'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:31:24.253432 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-fd8337eb-4d41-464a-b604-1c44b53ddcc9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a6e5b9c2c2c4455c8ff63113730f3a6f'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:31:24.260459 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-fd8337eb-4d41-464a-b604-1c44b53ddcc9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a6e5b9c2c2c4455c8ff63113730f3a6f'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:31:24.266799 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1361/2749] 10.222.0.15 () {58 vars in 1224 bytes} [Thu Sep 15 12:31:24 2022] DELETE /identity/v3/users/a6e5b9c2c2c4455c8ff63113730f3a6f => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:31:24.692013 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e100bd-cd2f-4061-8cd2-63641ed9cee1 admin admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:31:24.694414 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-d9e100bd-cd2f-4061-8cd2-63641ed9cee1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:31:24.713585 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e100bd-cd2f-4061-8cd2-63641ed9cee1 admin admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:31:24.796548 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2eb9d90-5300-445c-bee7-9432064e1cd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:31:24.797496 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2eb9d90-5300-445c-bee7-9432064e1cd9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:31:24.797979 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2eb9d90-5300-445c-bee7-9432064e1cd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:31:24.798256 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2eb9d90-5300-445c-bee7-9432064e1cd9 admin admin] PATH_INFO: `/v3/projects/73f6f3290c3a4670b85a186afda49c02` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:31:24.800044 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2eb9d90-5300-445c-bee7-9432064e1cd9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=73f6f3290c3a4670b85a186afda49c02)` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:31:24.801714 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2eb9d90-5300-445c-bee7-9432064e1cd9 admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:31:24.862538 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-a2eb9d90-5300-445c-bee7-9432064e1cd9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '73f6f3290c3a4670b85a186afda49c02', 'request_id': 'req-a2eb9d90-5300-445c-bee7-9432064e1cd9'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:31:24.873347 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-a2eb9d90-5300-445c-bee7-9432064e1cd9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '73f6f3290c3a4670b85a186afda49c02', 'request_id': 'req-a2eb9d90-5300-445c-bee7-9432064e1cd9'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:31:24.880872 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-a2eb9d90-5300-445c-bee7-9432064e1cd9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '73f6f3290c3a4670b85a186afda49c02', 'request_id': 'req-a2eb9d90-5300-445c-bee7-9432064e1cd9'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:31:24.893611 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-a2eb9d90-5300-445c-bee7-9432064e1cd9 admin admin] The token cache is being invalidate because project 73f6f3290c3a4670b85a186afda49c02 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81449) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 12:31:24.893986 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-a2eb9d90-5300-445c-bee7-9432064e1cd9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:31:24.894913 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-a2eb9d90-5300-445c-bee7-9432064e1cd9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:31:24.896711 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1389/2750] 10.222.0.15 () {58 vars in 1233 bytes} [Thu Sep 15 12:31:24 2022] DELETE /identity/v3/projects/73f6f3290c3a4670b85a186afda49c02 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:31:24.902087 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f8fd69-29f6-484c-9da0-31bd1ada43b4 None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:31:24.902574 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f8fd69-29f6-484c-9da0-31bd1ada43b4 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:31:24.902679 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f8fd69-29f6-484c-9da0-31bd1ada43b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:31:24.928494 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-d0f8fd69-29f6-484c-9da0-31bd1ada43b4 None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:31:24.929747 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-d0f8fd69-29f6-484c-9da0-31bd1ada43b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:31:25.004091 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1362/2751] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 12:31:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:31:25.014399 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953f7b10-464b-4543-ae90-700ffc7e6023 None None] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:31:25.014975 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953f7b10-464b-4543-ae90-700ffc7e6023 None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:31:25.014975 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953f7b10-464b-4543-ae90-700ffc7e6023 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:31:25.036846 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.auth.core [None req-953f7b10-464b-4543-ae90-700ffc7e6023 None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81449) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:31:25.038096 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-953f7b10-464b-4543-ae90-700ffc7e6023 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:31:25.046517 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1390/2752] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 12:31:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:31:25.050902 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f8fd69-29f6-484c-9da0-31bd1ada43b4 None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:31:25.060804 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f8fd69-29f6-484c-9da0-31bd1ada43b4 None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:31:25.066777 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d2a379-2a49-42e9-b1c2-a6280e9f10b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:31:25.067635 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d2a379-2a49-42e9-b1c2-a6280e9f10b6 admin admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:31:25.067923 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d2a379-2a49-42e9-b1c2-a6280e9f10b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:31:25.068215 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d2a379-2a49-42e9-b1c2-a6280e9f10b6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:31:25.068881 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d2a379-2a49-42e9-b1c2-a6280e9f10b6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:31:25.069189 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d2a379-2a49-42e9-b1c2-a6280e9f10b6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:31:25.070822 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d2a379-2a49-42e9-b1c2-a6280e9f10b6 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:31:25.087069 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1363/2753] 10.222.0.15 () {58 vars in 1153 bytes} [Thu Sep 15 12:31:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:31:25.088486 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953f7b10-464b-4543-ae90-700ffc7e6023 None None] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:31:25.098198 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953f7b10-464b-4543-ae90-700ffc7e6023 None None] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:31:25.103398 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1e2679-60e8-4579-b8f1-059f02f2ebdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:31:25.104000 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1e2679-60e8-4579-b8f1-059f02f2ebdd admin admin] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:31:25.104277 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1e2679-60e8-4579-b8f1-059f02f2ebdd admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:31:25.104548 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1e2679-60e8-4579-b8f1-059f02f2ebdd admin admin] PATH_INFO: `/v3/projects` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:31:25.105339 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1e2679-60e8-4579-b8f1-059f02f2ebdd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:31:25.107230 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1e2679-60e8-4579-b8f1-059f02f2ebdd admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:31:25.140387 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1391/2754] 10.222.0.15 () {60 vars in 1153 bytes} [Thu Sep 15 12:31:25 2022] POST /identity/v3/projects => generated 365 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 12:31:25.144889 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d2a379-2a49-42e9-b1c2-a6280e9f10b6 admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:31:25.161884 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d2a379-2a49-42e9-b1c2-a6280e9f10b6 admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:31:25.226351 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd31416-f558-4388-bf42-dc4f8f6dc350 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:31:25.227229 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd31416-f558-4388-bf42-dc4f8f6dc350 admin admin] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:31:25.227411 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd31416-f558-4388-bf42-dc4f8f6dc350 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:31:25.227556 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd31416-f558-4388-bf42-dc4f8f6dc350 admin admin] PATH_INFO: `/v3/users` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:31:25.228196 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd31416-f558-4388-bf42-dc4f8f6dc350 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:31:25.230140 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd31416-f558-4388-bf42-dc4f8f6dc350 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:31:25.259146 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1364/2755] 10.222.0.15 () {60 vars in 1144 bytes} [Thu Sep 15 12:31:25 2022] POST /identity/v3/users => generated 283 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 15 12:31:25.260354 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1e2679-60e8-4579-b8f1-059f02f2ebdd admin admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:31:25.271485 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1e2679-60e8-4579-b8f1-059f02f2ebdd admin admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:31:25.276695 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b434936-a7b1-40b1-b9c1-5a189f1ca0e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:31:25.277575 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b434936-a7b1-40b1-b9c1-5a189f1ca0e7 admin admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:31:25.278042 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b434936-a7b1-40b1-b9c1-5a189f1ca0e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:31:25.278819 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b434936-a7b1-40b1-b9c1-5a189f1ca0e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:31:25.279483 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b434936-a7b1-40b1-b9c1-5a189f1ca0e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:31:25.281549 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b434936-a7b1-40b1-b9c1-5a189f1ca0e7 admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:31:25.296968 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1392/2756] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 12:31:25 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 12:31:25.301199 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd31416-f558-4388-bf42-dc4f8f6dc350 admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:31:25.311472 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd31416-f558-4388-bf42-dc4f8f6dc350 admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:31:25.316932 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090ed867-da72-476c-accb-1c06853a5ae3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:31:25.317548 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090ed867-da72-476c-accb-1c06853a5ae3 admin admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:31:25.317829 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090ed867-da72-476c-accb-1c06853a5ae3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:31:25.318134 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090ed867-da72-476c-accb-1c06853a5ae3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:31:25.318771 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090ed867-da72-476c-accb-1c06853a5ae3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:31:25.320363 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090ed867-da72-476c-accb-1c06853a5ae3 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:31:25.335902 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1365/2757] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 12:31:25 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 15 12:31:25.340180 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b434936-a7b1-40b1-b9c1-5a189f1ca0e7 admin admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:31:25.350182 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b434936-a7b1-40b1-b9c1-5a189f1ca0e7 admin admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:31:25.355320 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3803a7c8-eb39-4f7a-aa5e-75eefbbf2078 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:31:25.355980 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3803a7c8-eb39-4f7a-aa5e-75eefbbf2078 admin admin] REQUEST_METHOD: `PUT` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:31:25.356259 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3803a7c8-eb39-4f7a-aa5e-75eefbbf2078 admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:31:25.356533 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3803a7c8-eb39-4f7a-aa5e-75eefbbf2078 admin admin] PATH_INFO: `/v3/projects/cdcc8573ba58467f90a5ae3d41e2b009/users/41c8d7a401114b46a545583d19686642/roles/0679d6628a25438d90dc4bdc0401eda2` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:31:25.374455 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3803a7c8-eb39-4f7a-aa5e-75eefbbf2078 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cdcc8573ba58467f90a5ae3d41e2b009, user_id=41c8d7a401114b46a545583d19686642, role_id=0679d6628a25438d90dc4bdc0401eda2)` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:31:25.377712 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3803a7c8-eb39-4f7a-aa5e-75eefbbf2078 admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:31:25.406611 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1393/2758] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 15 12:31:25 2022] PUT /identity/v3/projects/cdcc8573ba58467f90a5ae3d41e2b009/users/41c8d7a401114b46a545583d19686642/roles/0679d6628a25438d90dc4bdc0401eda2 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:31:25.408200 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ac176c-5e5e-49eb-b35f-30263bb4197a None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:31:25.408489 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ac176c-5e5e-49eb-b35f-30263bb4197a None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:31:25.408786 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ac176c-5e5e-49eb-b35f-30263bb4197a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:31:25.429966 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:31:25.449540 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-51ac176c-5e5e-49eb-b35f-30263bb4197a None None] MFA Rules not processed for user `41c8d7a401114b46a545583d19686642`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:31:25.451892 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-51ac176c-5e5e-49eb-b35f-30263bb4197a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:31:25.538980 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1366/2759] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 12:31:25 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:31:25.550404 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3803a7c8-eb39-4f7a-aa5e-75eefbbf2078 admin admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:31:25.552476 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-3803a7c8-eb39-4f7a-aa5e-75eefbbf2078 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:31:25.590839 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:31:25.628124 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3803a7c8-eb39-4f7a-aa5e-75eefbbf2078 admin admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:31:25.631866 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599477d5-2af7-4848-8279-a2bcebe235c8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8191d3ccb874529b32f8d0026db629d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '540a32ef88764e8c8d9201ae22f6ddaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:31:25.632646 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599477d5-2af7-4848-8279-a2bcebe235c8 service cinder] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:31:25.633001 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599477d5-2af7-4848-8279-a2bcebe235c8 service cinder] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:31:25.633273 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599477d5-2af7-4848-8279-a2bcebe235c8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:31:25.645051 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599477d5-2af7-4848-8279-a2bcebe235c8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:31:25.646426 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599477d5-2af7-4848-8279-a2bcebe235c8 service cinder] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:31:25.662129 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1394/2760] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 12:31:25 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 12:31:28.894059 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ac176c-5e5e-49eb-b35f-30263bb4197a None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:31:28.895531 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-51ac176c-5e5e-49eb-b35f-30263bb4197a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:31:28.982465 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ac176c-5e5e-49eb-b35f-30263bb4197a None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:31:28.987651 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f18f032-c58a-4ba5-a271-d232a61d3706 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:31:28.988285 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f18f032-c58a-4ba5-a271-d232a61d3706 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:31:28.988569 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f18f032-c58a-4ba5-a271-d232a61d3706 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:31:28.988900 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f18f032-c58a-4ba5-a271-d232a61d3706 admin admin] PATH_INFO: `/v3/users/c777718862d644e29b358ea28ad50cf7` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:31:28.990035 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f18f032-c58a-4ba5-a271-d232a61d3706 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c777718862d644e29b358ea28ad50cf7)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:31:28.991933 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f18f032-c58a-4ba5-a271-d232a61d3706 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:31:29.038450 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-5f18f032-c58a-4ba5-a271-d232a61d3706 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c777718862d644e29b358ea28ad50cf7'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:31:29.050674 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-5f18f032-c58a-4ba5-a271-d232a61d3706 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c777718862d644e29b358ea28ad50cf7'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:31:29.058169 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-5f18f032-c58a-4ba5-a271-d232a61d3706 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c777718862d644e29b358ea28ad50cf7'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:31:29.058903 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-5f18f032-c58a-4ba5-a271-d232a61d3706 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c777718862d644e29b358ea28ad50cf7'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:31:29.065823 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-5f18f032-c58a-4ba5-a271-d232a61d3706 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c777718862d644e29b358ea28ad50cf7'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:31:29.071721 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1367/2761] 10.222.0.15 () {58 vars in 1224 bytes} [Thu Sep 15 12:31:28 2022] DELETE /identity/v3/users/c777718862d644e29b358ea28ad50cf7 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:31:29.258217 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599477d5-2af7-4848-8279-a2bcebe235c8 service cinder] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:31:29.271284 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599477d5-2af7-4848-8279-a2bcebe235c8 service cinder] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:31:29.332391 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c24a6d-1f16-4a41-9708-8cd8238add91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:31:29.333057 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c24a6d-1f16-4a41-9708-8cd8238add91 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:31:29.333334 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c24a6d-1f16-4a41-9708-8cd8238add91 admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:31:29.333603 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c24a6d-1f16-4a41-9708-8cd8238add91 admin admin] PATH_INFO: `/v3/projects/0811b8305b9641aa884b0eb42a16e65b` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:31:29.334988 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c24a6d-1f16-4a41-9708-8cd8238add91 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0811b8305b9641aa884b0eb42a16e65b)` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:31:29.336897 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c24a6d-1f16-4a41-9708-8cd8238add91 admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:31:29.386019 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-a3c24a6d-1f16-4a41-9708-8cd8238add91 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0811b8305b9641aa884b0eb42a16e65b', 'request_id': 'req-a3c24a6d-1f16-4a41-9708-8cd8238add91'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:31:29.393640 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-a3c24a6d-1f16-4a41-9708-8cd8238add91 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0811b8305b9641aa884b0eb42a16e65b', 'request_id': 'req-a3c24a6d-1f16-4a41-9708-8cd8238add91'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:31:29.399120 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-a3c24a6d-1f16-4a41-9708-8cd8238add91 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0811b8305b9641aa884b0eb42a16e65b', 'request_id': 'req-a3c24a6d-1f16-4a41-9708-8cd8238add91'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:31:29.411160 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-a3c24a6d-1f16-4a41-9708-8cd8238add91 admin admin] The token cache is being invalidate because project 0811b8305b9641aa884b0eb42a16e65b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81449) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 12:31:29.411456 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-a3c24a6d-1f16-4a41-9708-8cd8238add91 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:31:29.412089 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-a3c24a6d-1f16-4a41-9708-8cd8238add91 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:31:29.416807 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1395/2762] 10.222.0.15 () {58 vars in 1233 bytes} [Thu Sep 15 12:31:29 2022] DELETE /identity/v3/projects/0811b8305b9641aa884b0eb42a16e65b => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:31:29.417667 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f18f032-c58a-4ba5-a271-d232a61d3706 admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:31:29.419211 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-5f18f032-c58a-4ba5-a271-d232a61d3706 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:31:29.432009 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f18f032-c58a-4ba5-a271-d232a61d3706 admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:31:29.444136 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:31:29.491163 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb03228-be33-4f40-9e3d-88f6dc180895 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:31:29.491808 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb03228-be33-4f40-9e3d-88f6dc180895 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:31:29.492083 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb03228-be33-4f40-9e3d-88f6dc180895 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:31:29.492350 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb03228-be33-4f40-9e3d-88f6dc180895 admin admin] PATH_INFO: `/v3/users/e81c271044734a5bb543d19197bb4dd5` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:31:29.493659 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb03228-be33-4f40-9e3d-88f6dc180895 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e81c271044734a5bb543d19197bb4dd5)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:31:29.495601 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb03228-be33-4f40-9e3d-88f6dc180895 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:31:29.545806 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-ceb03228-be33-4f40-9e3d-88f6dc180895 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e81c271044734a5bb543d19197bb4dd5'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:31:29.559665 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-ceb03228-be33-4f40-9e3d-88f6dc180895 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e81c271044734a5bb543d19197bb4dd5'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:31:29.568343 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-ceb03228-be33-4f40-9e3d-88f6dc180895 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e81c271044734a5bb543d19197bb4dd5'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:31:29.569252 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-ceb03228-be33-4f40-9e3d-88f6dc180895 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e81c271044734a5bb543d19197bb4dd5'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:31:29.576642 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-ceb03228-be33-4f40-9e3d-88f6dc180895 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e81c271044734a5bb543d19197bb4dd5'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:31:29.582509 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1368/2763] 10.222.0.15 () {58 vars in 1224 bytes} [Thu Sep 15 12:31:29 2022] DELETE /identity/v3/users/e81c271044734a5bb543d19197bb4dd5 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:31:29.758691 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c24a6d-1f16-4a41-9708-8cd8238add91 admin admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:31:29.760137 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-a3c24a6d-1f16-4a41-9708-8cd8238add91 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:31:29.776695 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c24a6d-1f16-4a41-9708-8cd8238add91 admin admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:31:29.832623 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-870e0ac9-aedb-4d2a-9a9d-af5daf29d65f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:31:29.833296 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870e0ac9-aedb-4d2a-9a9d-af5daf29d65f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:31:29.833570 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870e0ac9-aedb-4d2a-9a9d-af5daf29d65f admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:31:29.833837 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-870e0ac9-aedb-4d2a-9a9d-af5daf29d65f admin admin] PATH_INFO: `/v3/projects/fe2c3e20b5f64a1599fccbf437c65000` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:31:29.835192 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870e0ac9-aedb-4d2a-9a9d-af5daf29d65f admin admin] RBAC: Authorizing `identity:delete_project(project_id=fe2c3e20b5f64a1599fccbf437c65000)` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:31:29.837225 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-870e0ac9-aedb-4d2a-9a9d-af5daf29d65f admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:31:29.838605 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:31:29.884073 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-870e0ac9-aedb-4d2a-9a9d-af5daf29d65f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fe2c3e20b5f64a1599fccbf437c65000', 'request_id': 'req-870e0ac9-aedb-4d2a-9a9d-af5daf29d65f'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:31:29.890444 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-870e0ac9-aedb-4d2a-9a9d-af5daf29d65f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fe2c3e20b5f64a1599fccbf437c65000', 'request_id': 'req-870e0ac9-aedb-4d2a-9a9d-af5daf29d65f'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:31:29.896003 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-870e0ac9-aedb-4d2a-9a9d-af5daf29d65f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fe2c3e20b5f64a1599fccbf437c65000', 'request_id': 'req-870e0ac9-aedb-4d2a-9a9d-af5daf29d65f'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:31:29.906942 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-870e0ac9-aedb-4d2a-9a9d-af5daf29d65f admin admin] The token cache is being invalidate because project fe2c3e20b5f64a1599fccbf437c65000 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81449) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 12:31:29.907281 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-870e0ac9-aedb-4d2a-9a9d-af5daf29d65f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:31:29.908479 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-870e0ac9-aedb-4d2a-9a9d-af5daf29d65f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:31:29.909930 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1396/2764] 10.222.0.15 () {58 vars in 1233 bytes} [Thu Sep 15 12:31:29 2022] DELETE /identity/v3/projects/fe2c3e20b5f64a1599fccbf437c65000 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:31:29.914779 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9af20dd-1380-40c0-b779-1f315afee242 None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:31:29.915237 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9af20dd-1380-40c0-b779-1f315afee242 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:31:29.915332 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9af20dd-1380-40c0-b779-1f315afee242 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:31:29.934347 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-b9af20dd-1380-40c0-b779-1f315afee242 None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:31:29.935534 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-b9af20dd-1380-40c0-b779-1f315afee242 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:31:29.994282 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1369/2765] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 12:31:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:31:30.002211 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20091b3f-87da-4320-8fe0-848ebd4605e7 None None] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:31:30.002492 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20091b3f-87da-4320-8fe0-848ebd4605e7 None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:31:30.002763 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20091b3f-87da-4320-8fe0-848ebd4605e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:31:30.022661 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.auth.core [None req-20091b3f-87da-4320-8fe0-848ebd4605e7 None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81449) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:31:30.023846 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-20091b3f-87da-4320-8fe0-848ebd4605e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:31:30.031633 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1397/2766] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 12:31:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:31:30.036420 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9af20dd-1380-40c0-b779-1f315afee242 None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:31:30.046208 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9af20dd-1380-40c0-b779-1f315afee242 None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:31:30.050789 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6da3555-fcac-4be7-b4c8-d55de99eeb93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:31:30.051396 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6da3555-fcac-4be7-b4c8-d55de99eeb93 admin admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:31:30.051679 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6da3555-fcac-4be7-b4c8-d55de99eeb93 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:31:30.051953 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6da3555-fcac-4be7-b4c8-d55de99eeb93 admin admin] PATH_INFO: `/v3/domains` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:31:30.052596 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6da3555-fcac-4be7-b4c8-d55de99eeb93 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:31:30.052938 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6da3555-fcac-4be7-b4c8-d55de99eeb93 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:31:30.054525 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6da3555-fcac-4be7-b4c8-d55de99eeb93 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:31:30.066160 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1370/2767] 10.222.0.15 () {58 vars in 1153 bytes} [Thu Sep 15 12:31:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:31:30.070412 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20091b3f-87da-4320-8fe0-848ebd4605e7 None None] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:31:30.080976 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20091b3f-87da-4320-8fe0-848ebd4605e7 None None] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:31:30.086128 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7708db6e-e1dd-4a36-bec2-96b8263b0a5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:31:30.086736 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7708db6e-e1dd-4a36-bec2-96b8263b0a5f admin admin] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:31:30.087013 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7708db6e-e1dd-4a36-bec2-96b8263b0a5f admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:31:30.087283 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7708db6e-e1dd-4a36-bec2-96b8263b0a5f admin admin] PATH_INFO: `/v3/projects` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:31:30.087913 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7708db6e-e1dd-4a36-bec2-96b8263b0a5f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:31:30.089847 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7708db6e-e1dd-4a36-bec2-96b8263b0a5f admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:31:30.119315 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1398/2768] 10.222.0.15 () {60 vars in 1153 bytes} [Thu Sep 15 12:31:30 2022] POST /identity/v3/projects => generated 381 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 12:31:30.123544 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6da3555-fcac-4be7-b4c8-d55de99eeb93 admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:31:30.134255 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6da3555-fcac-4be7-b4c8-d55de99eeb93 admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:31:30.156699 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:31:30.194845 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfb5f98-d9ec-4879-9d29-688c900ee18b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:31:30.195473 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfb5f98-d9ec-4879-9d29-688c900ee18b admin admin] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:31:30.195780 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfb5f98-d9ec-4879-9d29-688c900ee18b admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:31:30.196022 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfb5f98-d9ec-4879-9d29-688c900ee18b admin admin] PATH_INFO: `/v3/users` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:31:30.196653 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dfb5f98-d9ec-4879-9d29-688c900ee18b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:31:30.198636 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dfb5f98-d9ec-4879-9d29-688c900ee18b admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:31:30.220333 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1371/2769] 10.222.0.15 () {60 vars in 1144 bytes} [Thu Sep 15 12:31:30 2022] POST /identity/v3/users => generated 291 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 15 12:31:30.224863 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7708db6e-e1dd-4a36-bec2-96b8263b0a5f admin admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:31:30.235809 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7708db6e-e1dd-4a36-bec2-96b8263b0a5f admin admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:31:30.241852 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5162f5c2-e9c0-480e-9fca-70fed0250ef3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:31:30.242530 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5162f5c2-e9c0-480e-9fca-70fed0250ef3 admin admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:31:30.242881 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5162f5c2-e9c0-480e-9fca-70fed0250ef3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:31:30.243253 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5162f5c2-e9c0-480e-9fca-70fed0250ef3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:31:30.243921 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5162f5c2-e9c0-480e-9fca-70fed0250ef3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:31:30.245631 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5162f5c2-e9c0-480e-9fca-70fed0250ef3 admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:31:30.257133 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1399/2770] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 12:31:30 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 12:31:30.261082 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfb5f98-d9ec-4879-9d29-688c900ee18b admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:31:30.271099 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfb5f98-d9ec-4879-9d29-688c900ee18b admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:31:30.275719 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8653d1c-2067-4874-8f1b-7e9451a944eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:31:30.276313 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8653d1c-2067-4874-8f1b-7e9451a944eb admin admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:31:30.276585 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8653d1c-2067-4874-8f1b-7e9451a944eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:31:30.276887 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8653d1c-2067-4874-8f1b-7e9451a944eb admin admin] PATH_INFO: `/v3/roles` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:31:30.277510 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8653d1c-2067-4874-8f1b-7e9451a944eb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:31:30.279201 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8653d1c-2067-4874-8f1b-7e9451a944eb admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:31:30.290457 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1372/2771] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 12:31:30 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 12:31:30.294953 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5162f5c2-e9c0-480e-9fca-70fed0250ef3 admin admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:31:30.305532 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5162f5c2-e9c0-480e-9fca-70fed0250ef3 admin admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:31:30.310284 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2800c66-357d-4bce-a765-6755b9d07e55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:31:30.310926 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2800c66-357d-4bce-a765-6755b9d07e55 admin admin] REQUEST_METHOD: `PUT` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:31:30.311201 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2800c66-357d-4bce-a765-6755b9d07e55 admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:31:30.311466 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2800c66-357d-4bce-a765-6755b9d07e55 admin admin] PATH_INFO: `/v3/projects/daddeef840a04bb8bc0d7b1f8cb37c7b/users/f0a002a6c60e4666982e1cc74ed90573/roles/0679d6628a25438d90dc4bdc0401eda2` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:31:30.327483 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2800c66-357d-4bce-a765-6755b9d07e55 admin admin] RBAC: Authorizing `identity:create_grant(project_id=daddeef840a04bb8bc0d7b1f8cb37c7b, user_id=f0a002a6c60e4666982e1cc74ed90573, role_id=0679d6628a25438d90dc4bdc0401eda2)` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:31:30.330257 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2800c66-357d-4bce-a765-6755b9d07e55 admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:31:30.343024 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1400/2772] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 15 12:31:30 2022] PUT /identity/v3/projects/daddeef840a04bb8bc0d7b1f8cb37c7b/users/f0a002a6c60e4666982e1cc74ed90573/roles/0679d6628a25438d90dc4bdc0401eda2 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:31:30.348601 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a020661-8fda-479c-82ae-773576776cad None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:31:30.348943 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a020661-8fda-479c-82ae-773576776cad None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:31:30.349214 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a020661-8fda-479c-82ae-773576776cad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:31:30.368770 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-9a020661-8fda-479c-82ae-773576776cad None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:31:30.369966 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-9a020661-8fda-479c-82ae-773576776cad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:31:30.428107 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1373/2773] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 12:31:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:31:30.437740 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2800c66-357d-4bce-a765-6755b9d07e55 admin admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:31:30.439543 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-a2800c66-357d-4bce-a765-6755b9d07e55 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:31:30.502912 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2800c66-357d-4bce-a765-6755b9d07e55 admin admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:31:30.507505 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d514bd7-75ba-40a4-991b-983a8fa6dc04 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfbb4d6809064f9cb9e06626ed7d6814', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '540a32ef88764e8c8d9201ae22f6ddaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:31:30.508845 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d514bd7-75ba-40a4-991b-983a8fa6dc04 service neutron] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:31:30.509132 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d514bd7-75ba-40a4-991b-983a8fa6dc04 service neutron] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:31:30.509406 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d514bd7-75ba-40a4-991b-983a8fa6dc04 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:31:30.518356 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d514bd7-75ba-40a4-991b-983a8fa6dc04 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:31:30.520098 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d514bd7-75ba-40a4-991b-983a8fa6dc04 service neutron] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:31:30.530641 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1401/2774] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 12:31:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 12:31:31.589582 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67630c6-ef2d-4578-b870-c55559ee79b9 None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:31:31.590142 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67630c6-ef2d-4578-b870-c55559ee79b9 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:31:31.590274 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67630c6-ef2d-4578-b870-c55559ee79b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:31:31.619175 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-d67630c6-ef2d-4578-b870-c55559ee79b9 None None] MFA Rules not processed for user `f0a002a6c60e4666982e1cc74ed90573`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:31:31.620554 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-d67630c6-ef2d-4578-b870-c55559ee79b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:31:31.689930 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1374/2775] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 12:31:31 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:31:31.703548 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d514bd7-75ba-40a4-991b-983a8fa6dc04 service neutron] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:31:31.705514 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-5d514bd7-75ba-40a4-991b-983a8fa6dc04 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:31:31.739716 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:31:31.768114 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d514bd7-75ba-40a4-991b-983a8fa6dc04 service neutron] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:31:31.771802 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ff6c77-c111-4ae8-8c88-10b1c5bd4db1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8191d3ccb874529b32f8d0026db629d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '540a32ef88764e8c8d9201ae22f6ddaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:31:31.772414 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ff6c77-c111-4ae8-8c88-10b1c5bd4db1 service cinder] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:31:31.772692 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ff6c77-c111-4ae8-8c88-10b1c5bd4db1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:31:31.773013 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ff6c77-c111-4ae8-8c88-10b1c5bd4db1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:31:31.781613 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ff6c77-c111-4ae8-8c88-10b1c5bd4db1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:31:31.783118 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ff6c77-c111-4ae8-8c88-10b1c5bd4db1 service cinder] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:31:31.791932 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1402/2776] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 12:31:31 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 15 12:31:36.308929 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b429987-24fb-4da1-8d10-a9176cb15015 None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:31:36.309420 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b429987-24fb-4da1-8d10-a9176cb15015 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:31:36.309420 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b429987-24fb-4da1-8d10-a9176cb15015 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:31:36.327778 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-5b429987-24fb-4da1-8d10-a9176cb15015 None None] MFA Rules not processed for user `f0a002a6c60e4666982e1cc74ed90573`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:31:36.328996 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-5b429987-24fb-4da1-8d10-a9176cb15015 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:31:36.336079 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1375/2777] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 12:31:36 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:31:41.787968 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=13 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:31:44.284079 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22301992-768d-4293-9eae-af5a39c5a307 None None] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:31:44.284705 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22301992-768d-4293-9eae-af5a39c5a307 None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:31:44.284705 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22301992-768d-4293-9eae-af5a39c5a307 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:31:44.306442 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.auth.core [None req-22301992-768d-4293-9eae-af5a39c5a307 None None] MFA Rules not processed for user `527f4dd6031546c785bc6360dacb60af`. Rule list: `[]` (Enabled: `True`). {{(pid=81449) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:31:44.308232 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-22301992-768d-4293-9eae-af5a39c5a307 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:31:44.373895 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1403/2778] 10.222.0.15 () {58 vars in 1009 bytes} [Thu Sep 15 12:31:44 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:31:44.390303 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b429987-24fb-4da1-8d10-a9176cb15015 None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:31:44.393098 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-5b429987-24fb-4da1-8d10-a9176cb15015 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:31:44.436648 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:31:44.483888 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b429987-24fb-4da1-8d10-a9176cb15015 None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:31:44.491324 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513360f6-d9ab-4069-bded-1ea2aecc12a2 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa0d3dc5b7540d59fdd59bbf64a04b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '540a32ef88764e8c8d9201ae22f6ddaf', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:31:44.491948 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513360f6-d9ab-4069-bded-1ea2aecc12a2 service swift] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:31:44.492987 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513360f6-d9ab-4069-bded-1ea2aecc12a2 service swift] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:31:44.493281 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-513360f6-d9ab-4069-bded-1ea2aecc12a2 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:31:44.505961 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513360f6-d9ab-4069-bded-1ea2aecc12a2 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:31:44.507539 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-513360f6-d9ab-4069-bded-1ea2aecc12a2 service swift] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:31:44.517672 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1376/2779] 10.222.0.15 () {60 vars in 1384 bytes} [Thu Sep 15 12:31:44 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 15 12:31:45.094948 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9463755b-2ff8-4907-8c3a-66e5e732ec27 None None] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:31:45.095266 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9463755b-2ff8-4907-8c3a-66e5e732ec27 None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:31:45.095568 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9463755b-2ff8-4907-8c3a-66e5e732ec27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:31:45.115275 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.auth.core [None req-9463755b-2ff8-4907-8c3a-66e5e732ec27 None None] MFA Rules not processed for user `527f4dd6031546c785bc6360dacb60af`. Rule list: `[]` (Enabled: `True`). {{(pid=81449) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:31:45.116516 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-9463755b-2ff8-4907-8c3a-66e5e732ec27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:31:45.123719 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1404/2780] 10.222.0.15 () {58 vars in 967 bytes} [Thu Sep 15 12:31:45 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:31:45.136271 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513360f6-d9ab-4069-bded-1ea2aecc12a2 service swift] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:31:45.146599 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-513360f6-d9ab-4069-bded-1ea2aecc12a2 service swift] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:31:45.151525 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a742452-6476-43b7-9213-595b5e9b28f5 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa0d3dc5b7540d59fdd59bbf64a04b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '540a32ef88764e8c8d9201ae22f6ddaf', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:31:45.152126 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a742452-6476-43b7-9213-595b5e9b28f5 service swift] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:31:45.152404 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a742452-6476-43b7-9213-595b5e9b28f5 service swift] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:31:45.152671 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a742452-6476-43b7-9213-595b5e9b28f5 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:31:45.160833 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a742452-6476-43b7-9213-595b5e9b28f5 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:31:45.162381 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a742452-6476-43b7-9213-595b5e9b28f5 service swift] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:31:45.170994 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1377/2781] 10.222.0.15 () {60 vars in 1384 bytes} [Thu Sep 15 12:31:45 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 15 12:31:45.435319 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90beb873-c863-4eef-8648-c8bc920758df None None] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:31:45.435847 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90beb873-c863-4eef-8648-c8bc920758df None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:31:45.436109 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90beb873-c863-4eef-8648-c8bc920758df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:31:45.457110 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.auth.core [None req-90beb873-c863-4eef-8648-c8bc920758df None None] MFA Rules not processed for user `527f4dd6031546c785bc6360dacb60af`. Rule list: `[]` (Enabled: `True`). {{(pid=81449) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:31:45.458934 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-90beb873-c863-4eef-8648-c8bc920758df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:31:45.467076 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1405/2782] 10.222.0.15 () {58 vars in 1009 bytes} [Thu Sep 15 12:31:45 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:31:45.479480 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a742452-6476-43b7-9213-595b5e9b28f5 service swift] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:31:45.490906 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a742452-6476-43b7-9213-595b5e9b28f5 service swift] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:31:45.497062 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da364d2f-e418-47f9-8dc2-8cae199d0ca6 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa0d3dc5b7540d59fdd59bbf64a04b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '540a32ef88764e8c8d9201ae22f6ddaf', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:31:45.497674 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da364d2f-e418-47f9-8dc2-8cae199d0ca6 service swift] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:31:45.497957 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da364d2f-e418-47f9-8dc2-8cae199d0ca6 service swift] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:31:45.498222 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da364d2f-e418-47f9-8dc2-8cae199d0ca6 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:31:45.507300 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da364d2f-e418-47f9-8dc2-8cae199d0ca6 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:31:45.509396 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da364d2f-e418-47f9-8dc2-8cae199d0ca6 service swift] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:31:45.519401 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1378/2783] 10.222.0.15 () {60 vars in 1384 bytes} [Thu Sep 15 12:31:45 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 15 12:31:45.869180 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5211ab-3571-407c-b7ff-c2c8fa7c1351 None None] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:31:45.869665 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5211ab-3571-407c-b7ff-c2c8fa7c1351 None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:31:45.869946 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5211ab-3571-407c-b7ff-c2c8fa7c1351 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:31:45.889631 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.auth.core [None req-5f5211ab-3571-407c-b7ff-c2c8fa7c1351 None None] MFA Rules not processed for user `527f4dd6031546c785bc6360dacb60af`. Rule list: `[]` (Enabled: `True`). {{(pid=81449) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:31:45.890840 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-5f5211ab-3571-407c-b7ff-c2c8fa7c1351 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:31:45.898779 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1406/2784] 10.222.0.15 () {58 vars in 967 bytes} [Thu Sep 15 12:31:45 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:31:45.912631 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da364d2f-e418-47f9-8dc2-8cae199d0ca6 service swift] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:31:45.925497 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da364d2f-e418-47f9-8dc2-8cae199d0ca6 service swift] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:31:45.930669 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be63e37-8166-4bdb-a5c7-445d0443dfc3 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa0d3dc5b7540d59fdd59bbf64a04b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '540a32ef88764e8c8d9201ae22f6ddaf', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:31:45.931317 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be63e37-8166-4bdb-a5c7-445d0443dfc3 service swift] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:31:45.931602 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be63e37-8166-4bdb-a5c7-445d0443dfc3 service swift] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:31:45.931876 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be63e37-8166-4bdb-a5c7-445d0443dfc3 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:31:45.941052 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be63e37-8166-4bdb-a5c7-445d0443dfc3 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:31:45.942676 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be63e37-8166-4bdb-a5c7-445d0443dfc3 service swift] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:31:45.951383 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1379/2785] 10.222.0.15 () {60 vars in 1384 bytes} [Thu Sep 15 12:31:45 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 15 12:31:48.205460 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5211ab-3571-407c-b7ff-c2c8fa7c1351 None None] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:31:48.207908 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-5f5211ab-3571-407c-b7ff-c2c8fa7c1351 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:31:48.223362 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5211ab-3571-407c-b7ff-c2c8fa7c1351 None None] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:31:48.227900 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b645d05-7f09-4a55-add1-2d57e375d33e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:31:48.228526 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b645d05-7f09-4a55-add1-2d57e375d33e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:31:48.228825 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b645d05-7f09-4a55-add1-2d57e375d33e admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:31:48.229708 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b645d05-7f09-4a55-add1-2d57e375d33e admin admin] PATH_INFO: `/v3/users/b87556e083ae4854a4d26336490ccd1d` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:31:48.230857 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b645d05-7f09-4a55-add1-2d57e375d33e admin admin] RBAC: Authorizing `identity:delete_user(user_id=b87556e083ae4854a4d26336490ccd1d)` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:31:48.232777 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b645d05-7f09-4a55-add1-2d57e375d33e admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:31:48.288420 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-8b645d05-7f09-4a55-add1-2d57e375d33e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b87556e083ae4854a4d26336490ccd1d'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:31:48.300471 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-8b645d05-7f09-4a55-add1-2d57e375d33e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b87556e083ae4854a4d26336490ccd1d'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:31:48.315445 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-8b645d05-7f09-4a55-add1-2d57e375d33e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b87556e083ae4854a4d26336490ccd1d'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:31:48.316222 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-8b645d05-7f09-4a55-add1-2d57e375d33e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b87556e083ae4854a4d26336490ccd1d'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:31:48.328197 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-8b645d05-7f09-4a55-add1-2d57e375d33e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b87556e083ae4854a4d26336490ccd1d'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:31:48.335173 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1407/2786] 10.222.0.15 () {58 vars in 1224 bytes} [Thu Sep 15 12:31:48 2022] DELETE /identity/v3/users/b87556e083ae4854a4d26336490ccd1d => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:31:48.339616 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede1bbe4-b987-4c9f-8e3f-4e0afead2527 None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:31:48.340095 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede1bbe4-b987-4c9f-8e3f-4e0afead2527 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:31:48.340300 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede1bbe4-b987-4c9f-8e3f-4e0afead2527 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:31:48.361511 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-ede1bbe4-b987-4c9f-8e3f-4e0afead2527 None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:31:48.362710 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-ede1bbe4-b987-4c9f-8e3f-4e0afead2527 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:31:48.423212 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1380/2787] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 12:31:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:31:48.433426 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b645d05-7f09-4a55-add1-2d57e375d33e admin admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:31:48.435027 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-8b645d05-7f09-4a55-add1-2d57e375d33e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:31:48.497091 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b645d05-7f09-4a55-add1-2d57e375d33e admin admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:31:48.500696 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3842cbcb-8d0a-4eeb-ad3e-d730bf032dab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfbb4d6809064f9cb9e06626ed7d6814', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '540a32ef88764e8c8d9201ae22f6ddaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:31:48.501330 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3842cbcb-8d0a-4eeb-ad3e-d730bf032dab service neutron] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:31:48.501603 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3842cbcb-8d0a-4eeb-ad3e-d730bf032dab service neutron] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:31:48.501869 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3842cbcb-8d0a-4eeb-ad3e-d730bf032dab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:31:48.511838 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3842cbcb-8d0a-4eeb-ad3e-d730bf032dab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:31:48.513451 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3842cbcb-8d0a-4eeb-ad3e-d730bf032dab service neutron] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:31:48.518953 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:31:48.524941 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1408/2788] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 12:31:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 12:31:48.694272 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede1bbe4-b987-4c9f-8e3f-4e0afead2527 None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:31:48.696219 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-ede1bbe4-b987-4c9f-8e3f-4e0afead2527 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:31:48.716169 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede1bbe4-b987-4c9f-8e3f-4e0afead2527 None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:31:48.721854 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f112a4c2-7aef-47ac-9220-49e80779e78b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:31:48.722380 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f112a4c2-7aef-47ac-9220-49e80779e78b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:31:48.722655 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f112a4c2-7aef-47ac-9220-49e80779e78b admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:31:48.722928 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f112a4c2-7aef-47ac-9220-49e80779e78b admin admin] PATH_INFO: `/v3/projects/617b2e1cc38d4faabe0ac41504480cb4` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:31:48.724085 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f112a4c2-7aef-47ac-9220-49e80779e78b admin admin] RBAC: Authorizing `identity:delete_project(project_id=617b2e1cc38d4faabe0ac41504480cb4)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:31:48.725984 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f112a4c2-7aef-47ac-9220-49e80779e78b admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:31:48.776111 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-f112a4c2-7aef-47ac-9220-49e80779e78b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '617b2e1cc38d4faabe0ac41504480cb4', 'request_id': 'req-f112a4c2-7aef-47ac-9220-49e80779e78b'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:31:48.786646 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-f112a4c2-7aef-47ac-9220-49e80779e78b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '617b2e1cc38d4faabe0ac41504480cb4', 'request_id': 'req-f112a4c2-7aef-47ac-9220-49e80779e78b'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:31:48.792895 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-f112a4c2-7aef-47ac-9220-49e80779e78b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '617b2e1cc38d4faabe0ac41504480cb4', 'request_id': 'req-f112a4c2-7aef-47ac-9220-49e80779e78b'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:31:48.800493 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:31:48.805517 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-f112a4c2-7aef-47ac-9220-49e80779e78b admin admin] The token cache is being invalidate because project 617b2e1cc38d4faabe0ac41504480cb4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81448) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 12:31:48.805988 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-f112a4c2-7aef-47ac-9220-49e80779e78b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:31:48.807023 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-f112a4c2-7aef-47ac-9220-49e80779e78b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:31:48.808827 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1381/2789] 10.222.0.15 () {58 vars in 1233 bytes} [Thu Sep 15 12:31:48 2022] DELETE /identity/v3/projects/617b2e1cc38d4faabe0ac41504480cb4 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:31:48.813741 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e32f35b-1474-4e91-b5fa-650ea2c932e2 None None] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:31:48.814161 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e32f35b-1474-4e91-b5fa-650ea2c932e2 None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:31:48.814511 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e32f35b-1474-4e91-b5fa-650ea2c932e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:31:48.838646 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.auth.core [None req-9e32f35b-1474-4e91-b5fa-650ea2c932e2 None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81449) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:31:48.839864 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-9e32f35b-1474-4e91-b5fa-650ea2c932e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:31:48.899630 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1409/2790] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 12:31:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:31:48.907690 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84343d32-127d-42c2-b6a1-eaf5e1f874e8 None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:31:48.907974 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84343d32-127d-42c2-b6a1-eaf5e1f874e8 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:31:48.908250 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84343d32-127d-42c2-b6a1-eaf5e1f874e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:31:48.928161 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-84343d32-127d-42c2-b6a1-eaf5e1f874e8 None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:31:48.929384 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-84343d32-127d-42c2-b6a1-eaf5e1f874e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:31:48.937303 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1382/2791] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 12:31:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:31:48.941675 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e32f35b-1474-4e91-b5fa-650ea2c932e2 None None] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:31:48.952203 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e32f35b-1474-4e91-b5fa-650ea2c932e2 None None] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:31:48.957050 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a008bf-6976-44b2-9ef5-5673dc6cbf78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:31:48.957680 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a008bf-6976-44b2-9ef5-5673dc6cbf78 admin admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:31:48.958134 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a008bf-6976-44b2-9ef5-5673dc6cbf78 admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:31:48.958413 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a008bf-6976-44b2-9ef5-5673dc6cbf78 admin admin] PATH_INFO: `/v3/domains` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:31:48.959303 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a008bf-6976-44b2-9ef5-5673dc6cbf78 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81449) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:31:48.959600 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a008bf-6976-44b2-9ef5-5673dc6cbf78 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:31:48.961268 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a008bf-6976-44b2-9ef5-5673dc6cbf78 admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:31:48.973119 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1410/2792] 10.222.0.15 () {58 vars in 1153 bytes} [Thu Sep 15 12:31:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:31:48.979311 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84343d32-127d-42c2-b6a1-eaf5e1f874e8 None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:31:48.988951 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84343d32-127d-42c2-b6a1-eaf5e1f874e8 None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:31:48.994367 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d35b8c-9a71-4fa4-8354-a2e1b220821d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:31:48.995053 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d35b8c-9a71-4fa4-8354-a2e1b220821d admin admin] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:31:48.995393 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d35b8c-9a71-4fa4-8354-a2e1b220821d admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:31:48.995761 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d35b8c-9a71-4fa4-8354-a2e1b220821d admin admin] PATH_INFO: `/v3/projects` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:31:48.996460 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d35b8c-9a71-4fa4-8354-a2e1b220821d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:31:48.998451 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d35b8c-9a71-4fa4-8354-a2e1b220821d admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:31:49.026536 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1383/2793] 10.222.0.15 () {60 vars in 1153 bytes} [Thu Sep 15 12:31:48 2022] POST /identity/v3/projects => generated 391 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 12:31:49.030783 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a008bf-6976-44b2-9ef5-5673dc6cbf78 admin admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:31:49.041454 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a008bf-6976-44b2-9ef5-5673dc6cbf78 admin admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:31:49.102876 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320a9428-53a9-4c1f-8df3-4ea910cafb3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:31:49.103484 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320a9428-53a9-4c1f-8df3-4ea910cafb3f admin admin] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:31:49.103759 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320a9428-53a9-4c1f-8df3-4ea910cafb3f admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:31:49.104024 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-320a9428-53a9-4c1f-8df3-4ea910cafb3f admin admin] PATH_INFO: `/v3/users` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:31:49.104642 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-320a9428-53a9-4c1f-8df3-4ea910cafb3f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:31:49.106827 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-320a9428-53a9-4c1f-8df3-4ea910cafb3f admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:31:49.116841 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d35b8c-9a71-4fa4-8354-a2e1b220821d admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:31:49.119431 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-73d35b8c-9a71-4fa4-8354-a2e1b220821d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:31:49.129249 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1411/2794] 10.222.0.15 () {60 vars in 1144 bytes} [Thu Sep 15 12:31:49 2022] POST /identity/v3/users => generated 296 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 12:31:49.133123 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320a9428-53a9-4c1f-8df3-4ea910cafb3f admin admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:31:49.136090 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d35b8c-9a71-4fa4-8354-a2e1b220821d admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:31:49.140825 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516460f7-d81e-4990-b12c-8218f4a94d0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:31:49.141549 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516460f7-d81e-4990-b12c-8218f4a94d0b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:31:49.141834 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516460f7-d81e-4990-b12c-8218f4a94d0b admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:31:49.142106 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516460f7-d81e-4990-b12c-8218f4a94d0b admin admin] PATH_INFO: `/v3/users/e0c911f04b5f440896ded2c38b68b5d6` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:31:49.142284 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-320a9428-53a9-4c1f-8df3-4ea910cafb3f admin admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:31:49.143234 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-516460f7-d81e-4990-b12c-8218f4a94d0b admin admin] RBAC: Authorizing `identity:delete_user(user_id=e0c911f04b5f440896ded2c38b68b5d6)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:31:49.145324 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-516460f7-d81e-4990-b12c-8218f4a94d0b admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:31:49.146963 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6a52dd-9b93-4804-aace-107ba690b7cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:31:49.147559 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6a52dd-9b93-4804-aace-107ba690b7cf admin admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:31:49.147818 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6a52dd-9b93-4804-aace-107ba690b7cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:31:49.148086 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6a52dd-9b93-4804-aace-107ba690b7cf admin admin] PATH_INFO: `/v3/roles` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:31:49.149061 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6a52dd-9b93-4804-aace-107ba690b7cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:31:49.150823 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6a52dd-9b93-4804-aace-107ba690b7cf admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:31:49.162988 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1412/2795] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 12:31:49 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 15 12:31:49.166930 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6a52dd-9b93-4804-aace-107ba690b7cf admin admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:31:49.177453 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6a52dd-9b93-4804-aace-107ba690b7cf admin admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:31:49.182437 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e92b910-8594-4055-9fe3-b13e9f35df46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:31:49.183045 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e92b910-8594-4055-9fe3-b13e9f35df46 admin admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:31:49.183336 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e92b910-8594-4055-9fe3-b13e9f35df46 admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:31:49.183805 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e92b910-8594-4055-9fe3-b13e9f35df46 admin admin] PATH_INFO: `/v3/roles` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:31:49.185062 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e92b910-8594-4055-9fe3-b13e9f35df46 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:31:49.186636 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e92b910-8594-4055-9fe3-b13e9f35df46 admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:31:49.198442 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1413/2796] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 12:31:49 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 15 12:31:49.202425 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e92b910-8594-4055-9fe3-b13e9f35df46 admin admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:31:49.209385 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-516460f7-d81e-4990-b12c-8218f4a94d0b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e0c911f04b5f440896ded2c38b68b5d6'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:31:49.212387 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e92b910-8594-4055-9fe3-b13e9f35df46 admin admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:31:49.217179 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849c4e2f-6ab2-4750-8986-aa7da80cf7f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:31:49.217827 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849c4e2f-6ab2-4750-8986-aa7da80cf7f8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:31:49.218157 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849c4e2f-6ab2-4750-8986-aa7da80cf7f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:31:49.218438 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849c4e2f-6ab2-4750-8986-aa7da80cf7f8 admin admin] PATH_INFO: `/v3/projects/b36c45c35c664885b236c2c25189f63e/users/6af3b41a17a84e1e92349e36d830f961/roles/0679d6628a25438d90dc4bdc0401eda2` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:31:49.221752 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-516460f7-d81e-4990-b12c-8218f4a94d0b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e0c911f04b5f440896ded2c38b68b5d6'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:31:49.229457 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-516460f7-d81e-4990-b12c-8218f4a94d0b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e0c911f04b5f440896ded2c38b68b5d6'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:31:49.230139 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-516460f7-d81e-4990-b12c-8218f4a94d0b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e0c911f04b5f440896ded2c38b68b5d6'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:31:49.234856 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-849c4e2f-6ab2-4750-8986-aa7da80cf7f8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b36c45c35c664885b236c2c25189f63e, user_id=6af3b41a17a84e1e92349e36d830f961, role_id=0679d6628a25438d90dc4bdc0401eda2)` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:31:49.237558 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-849c4e2f-6ab2-4750-8986-aa7da80cf7f8 admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:31:49.249619 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-516460f7-d81e-4990-b12c-8218f4a94d0b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e0c911f04b5f440896ded2c38b68b5d6'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:31:49.252974 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1414/2797] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 15 12:31:49 2022] PUT /identity/v3/projects/b36c45c35c664885b236c2c25189f63e/users/6af3b41a17a84e1e92349e36d830f961/roles/0679d6628a25438d90dc4bdc0401eda2 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:31:49.263285 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e69a10-e641-4628-b8da-ee70db8c06af None None] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:31:49.263725 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e69a10-e641-4628-b8da-ee70db8c06af None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:31:49.264083 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1384/2798] 10.222.0.15 () {58 vars in 1224 bytes} [Thu Sep 15 12:31:49 2022] DELETE /identity/v3/users/e0c911f04b5f440896ded2c38b68b5d6 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 12:31:49.264230 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e69a10-e641-4628-b8da-ee70db8c06af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:31:49.268138 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1d3ea7-bcc3-4a08-85c9-06d99380c8d2 None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:31:49.268418 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1d3ea7-bcc3-4a08-85c9-06d99380c8d2 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:31:49.268686 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1d3ea7-bcc3-4a08-85c9-06d99380c8d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:31:49.293195 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-cf1d3ea7-bcc3-4a08-85c9-06d99380c8d2 None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:31:49.293837 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.auth.core [None req-55e69a10-e641-4628-b8da-ee70db8c06af None None] MFA Rules not processed for user `6af3b41a17a84e1e92349e36d830f961`. Rule list: `[]` (Enabled: `True`). {{(pid=81449) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:31:49.294454 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-cf1d3ea7-bcc3-4a08-85c9-06d99380c8d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:31:49.295288 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-55e69a10-e641-4628-b8da-ee70db8c06af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:31:49.315719 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:31:49.378360 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1385/2799] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 12:31:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 12:31:49.386160 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1415/2800] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 12:31:49 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 12:31:49.393852 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1d3ea7-bcc3-4a08-85c9-06d99380c8d2 None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:31:49.395437 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-cf1d3ea7-bcc3-4a08-85c9-06d99380c8d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:31:49.398485 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e69a10-e641-4628-b8da-ee70db8c06af None None] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:31:49.400841 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-55e69a10-e641-4628-b8da-ee70db8c06af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:31:49.461840 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:31:49.484609 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1d3ea7-bcc3-4a08-85c9-06d99380c8d2 None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:31:49.485017 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e69a10-e641-4628-b8da-ee70db8c06af None None] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:31:49.491253 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49ae01f-d028-4603-8f91-39faa0dc080a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfbb4d6809064f9cb9e06626ed7d6814', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '540a32ef88764e8c8d9201ae22f6ddaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:31:49.491885 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972d1bdb-52b6-4e60-8cc1-ccc86eb6e63e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8191d3ccb874529b32f8d0026db629d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '540a32ef88764e8c8d9201ae22f6ddaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:31:49.492603 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972d1bdb-52b6-4e60-8cc1-ccc86eb6e63e service cinder] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:31:49.492852 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49ae01f-d028-4603-8f91-39faa0dc080a service neutron] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:31:49.493096 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972d1bdb-52b6-4e60-8cc1-ccc86eb6e63e service cinder] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:31:49.493466 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49ae01f-d028-4603-8f91-39faa0dc080a service neutron] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:31:49.493594 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972d1bdb-52b6-4e60-8cc1-ccc86eb6e63e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:31:49.494048 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49ae01f-d028-4603-8f91-39faa0dc080a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:31:49.507511 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972d1bdb-52b6-4e60-8cc1-ccc86eb6e63e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:31:49.509596 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972d1bdb-52b6-4e60-8cc1-ccc86eb6e63e service cinder] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:31:49.518748 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49ae01f-d028-4603-8f91-39faa0dc080a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:31:49.520547 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49ae01f-d028-4603-8f91-39faa0dc080a service neutron] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:31:49.522677 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1416/2801] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 12:31:49 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 15 12:31:49.549712 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1386/2802] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 12:31:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 12:31:49.860846 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972d1bdb-52b6-4e60-8cc1-ccc86eb6e63e service cinder] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:31:49.862394 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-972d1bdb-52b6-4e60-8cc1-ccc86eb6e63e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:31:49.880982 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972d1bdb-52b6-4e60-8cc1-ccc86eb6e63e service cinder] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:31:49.888675 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23d201b-ac71-46e4-9c50-b52d302c4dd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:31:49.893022 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23d201b-ac71-46e4-9c50-b52d302c4dd2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:31:49.893422 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23d201b-ac71-46e4-9c50-b52d302c4dd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:31:49.893922 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23d201b-ac71-46e4-9c50-b52d302c4dd2 admin admin] PATH_INFO: `/v3/projects/e6064d75f39949199c149f111abcae66` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:31:49.896033 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23d201b-ac71-46e4-9c50-b52d302c4dd2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e6064d75f39949199c149f111abcae66)` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:31:49.898186 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23d201b-ac71-46e4-9c50-b52d302c4dd2 admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:31:49.965842 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-c23d201b-ac71-46e4-9c50-b52d302c4dd2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e6064d75f39949199c149f111abcae66', 'request_id': 'req-c23d201b-ac71-46e4-9c50-b52d302c4dd2'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:31:49.974659 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-c23d201b-ac71-46e4-9c50-b52d302c4dd2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e6064d75f39949199c149f111abcae66', 'request_id': 'req-c23d201b-ac71-46e4-9c50-b52d302c4dd2'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:31:49.981222 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-c23d201b-ac71-46e4-9c50-b52d302c4dd2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e6064d75f39949199c149f111abcae66', 'request_id': 'req-c23d201b-ac71-46e4-9c50-b52d302c4dd2'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:31:49.993021 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-c23d201b-ac71-46e4-9c50-b52d302c4dd2 admin admin] The token cache is being invalidate because project e6064d75f39949199c149f111abcae66 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81449) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 12:31:49.993309 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-c23d201b-ac71-46e4-9c50-b52d302c4dd2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:31:49.993953 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-c23d201b-ac71-46e4-9c50-b52d302c4dd2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:31:49.995557 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1417/2803] 10.222.0.15 () {58 vars in 1233 bytes} [Thu Sep 15 12:31:49 2022] DELETE /identity/v3/projects/e6064d75f39949199c149f111abcae66 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:31:59.527833 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=6 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:31:59.989469 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=71,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:32:09.037107 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49ae01f-d028-4603-8f91-39faa0dc080a service neutron] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:32:09.038532 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-d49ae01f-d028-4603-8f91-39faa0dc080a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:32:09.051261 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49ae01f-d028-4603-8f91-39faa0dc080a service neutron] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:32:09.109390 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af435afa-7d0a-45ee-b8dd-a3d52ec9224e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:32:09.110031 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af435afa-7d0a-45ee-b8dd-a3d52ec9224e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:32:09.110311 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af435afa-7d0a-45ee-b8dd-a3d52ec9224e admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:32:09.110582 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af435afa-7d0a-45ee-b8dd-a3d52ec9224e admin admin] PATH_INFO: `/v3/users/6af3b41a17a84e1e92349e36d830f961` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:32:09.111706 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af435afa-7d0a-45ee-b8dd-a3d52ec9224e admin admin] RBAC: Authorizing `identity:delete_user(user_id=6af3b41a17a84e1e92349e36d830f961)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:32:09.113694 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af435afa-7d0a-45ee-b8dd-a3d52ec9224e admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:32:09.159913 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-af435afa-7d0a-45ee-b8dd-a3d52ec9224e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6af3b41a17a84e1e92349e36d830f961'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:32:09.172665 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-af435afa-7d0a-45ee-b8dd-a3d52ec9224e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6af3b41a17a84e1e92349e36d830f961'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:32:09.179918 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-af435afa-7d0a-45ee-b8dd-a3d52ec9224e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6af3b41a17a84e1e92349e36d830f961'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:32:09.180599 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-af435afa-7d0a-45ee-b8dd-a3d52ec9224e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6af3b41a17a84e1e92349e36d830f961'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:32:09.187812 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-af435afa-7d0a-45ee-b8dd-a3d52ec9224e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6af3b41a17a84e1e92349e36d830f961'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:32:09.194406 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1387/2804] 10.222.0.15 () {58 vars in 1224 bytes} [Thu Sep 15 12:32:09 2022] DELETE /identity/v3/users/6af3b41a17a84e1e92349e36d830f961 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:32:09.198878 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e0e2a7-a593-4744-80f2-2212e32a415e None None] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:32:09.199340 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e0e2a7-a593-4744-80f2-2212e32a415e None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:32:09.199445 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e0e2a7-a593-4744-80f2-2212e32a415e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:32:09.222400 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.auth.core [None req-f8e0e2a7-a593-4744-80f2-2212e32a415e None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81449) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:32:09.222400 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-f8e0e2a7-a593-4744-80f2-2212e32a415e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:32:09.280492 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1418/2805] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 12:32:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 12:32:09.290769 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af435afa-7d0a-45ee-b8dd-a3d52ec9224e admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:32:09.292253 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-af435afa-7d0a-45ee-b8dd-a3d52ec9224e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:32:09.349081 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af435afa-7d0a-45ee-b8dd-a3d52ec9224e admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:32:09.352734 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59eb5ca3-a8a4-4b29-bb24-42fcce2ff398 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfbb4d6809064f9cb9e06626ed7d6814', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '540a32ef88764e8c8d9201ae22f6ddaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:32:09.353388 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59eb5ca3-a8a4-4b29-bb24-42fcce2ff398 service neutron] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:32:09.353686 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59eb5ca3-a8a4-4b29-bb24-42fcce2ff398 service neutron] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:32:09.354008 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59eb5ca3-a8a4-4b29-bb24-42fcce2ff398 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:32:09.363588 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59eb5ca3-a8a4-4b29-bb24-42fcce2ff398 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:32:09.365577 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59eb5ca3-a8a4-4b29-bb24-42fcce2ff398 service neutron] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:32:09.374803 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1388/2806] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 12:32:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 12:32:09.654602 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e0e2a7-a593-4744-80f2-2212e32a415e None None] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:32:09.656092 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-f8e0e2a7-a593-4744-80f2-2212e32a415e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:32:09.669375 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e0e2a7-a593-4744-80f2-2212e32a415e None None] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:32:09.674331 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1847b9-3fdf-4d0b-9137-706f07d3fd1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:32:09.674975 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1847b9-3fdf-4d0b-9137-706f07d3fd1a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:32:09.675256 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1847b9-3fdf-4d0b-9137-706f07d3fd1a admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:32:09.675559 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1847b9-3fdf-4d0b-9137-706f07d3fd1a admin admin] PATH_INFO: `/v3/projects/b36c45c35c664885b236c2c25189f63e` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:32:09.676707 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f1847b9-3fdf-4d0b-9137-706f07d3fd1a admin admin] RBAC: Authorizing `identity:delete_project(project_id=b36c45c35c664885b236c2c25189f63e)` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:32:09.678744 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f1847b9-3fdf-4d0b-9137-706f07d3fd1a admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:32:09.727896 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-8f1847b9-3fdf-4d0b-9137-706f07d3fd1a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b36c45c35c664885b236c2c25189f63e', 'request_id': 'req-8f1847b9-3fdf-4d0b-9137-706f07d3fd1a'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:32:09.736130 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-8f1847b9-3fdf-4d0b-9137-706f07d3fd1a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b36c45c35c664885b236c2c25189f63e', 'request_id': 'req-8f1847b9-3fdf-4d0b-9137-706f07d3fd1a'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:32:09.742129 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-8f1847b9-3fdf-4d0b-9137-706f07d3fd1a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b36c45c35c664885b236c2c25189f63e', 'request_id': 'req-8f1847b9-3fdf-4d0b-9137-706f07d3fd1a'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:32:09.753277 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-8f1847b9-3fdf-4d0b-9137-706f07d3fd1a admin admin] The token cache is being invalidate because project b36c45c35c664885b236c2c25189f63e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81449) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 12:32:09.753562 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-8f1847b9-3fdf-4d0b-9137-706f07d3fd1a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:32:09.754197 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-8f1847b9-3fdf-4d0b-9137-706f07d3fd1a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:32:09.755675 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1419/2807] 10.222.0.15 () {58 vars in 1233 bytes} [Thu Sep 15 12:32:09 2022] DELETE /identity/v3/projects/b36c45c35c664885b236c2c25189f63e => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:32:09.906929 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59eb5ca3-a8a4-4b29-bb24-42fcce2ff398 service neutron] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:32:09.908381 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-59eb5ca3-a8a4-4b29-bb24-42fcce2ff398 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:32:09.939539 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:32:10.014486 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59eb5ca3-a8a4-4b29-bb24-42fcce2ff398 service neutron] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:32:10.072880 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321c705b-1f65-4fa7-921b-f9c9bb15b382 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:32:10.073571 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321c705b-1f65-4fa7-921b-f9c9bb15b382 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:32:10.073852 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321c705b-1f65-4fa7-921b-f9c9bb15b382 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:32:10.074126 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321c705b-1f65-4fa7-921b-f9c9bb15b382 admin admin] PATH_INFO: `/v3/users/41c8d7a401114b46a545583d19686642` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:32:10.075356 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321c705b-1f65-4fa7-921b-f9c9bb15b382 admin admin] RBAC: Authorizing `identity:delete_user(user_id=41c8d7a401114b46a545583d19686642)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:32:10.077418 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321c705b-1f65-4fa7-921b-f9c9bb15b382 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:32:10.124714 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-321c705b-1f65-4fa7-921b-f9c9bb15b382 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '41c8d7a401114b46a545583d19686642'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:32:10.137062 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-321c705b-1f65-4fa7-921b-f9c9bb15b382 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '41c8d7a401114b46a545583d19686642'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:32:10.144415 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-321c705b-1f65-4fa7-921b-f9c9bb15b382 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '41c8d7a401114b46a545583d19686642'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:32:10.145090 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-321c705b-1f65-4fa7-921b-f9c9bb15b382 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '41c8d7a401114b46a545583d19686642'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:32:10.153464 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-321c705b-1f65-4fa7-921b-f9c9bb15b382 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '41c8d7a401114b46a545583d19686642'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:32:10.159407 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1389/2808] 10.222.0.15 () {58 vars in 1224 bytes} [Thu Sep 15 12:32:09 2022] DELETE /identity/v3/users/41c8d7a401114b46a545583d19686642 => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:32:10.163991 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a073e178-f7f3-4e92-ac27-ffb156c21fa7 None None] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:32:10.164280 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a073e178-f7f3-4e92-ac27-ffb156c21fa7 None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:32:10.164555 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a073e178-f7f3-4e92-ac27-ffb156c21fa7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:32:10.193518 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.auth.core [None req-a073e178-f7f3-4e92-ac27-ffb156c21fa7 None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81449) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:32:10.194773 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-a073e178-f7f3-4e92-ac27-ffb156c21fa7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:32:10.259369 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1420/2809] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 12:32:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 12:32:10.269301 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321c705b-1f65-4fa7-921b-f9c9bb15b382 admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:32:10.271306 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-321c705b-1f65-4fa7-921b-f9c9bb15b382 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:32:10.319750 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:32:10.351897 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321c705b-1f65-4fa7-921b-f9c9bb15b382 admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:32:10.355587 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1330bbb4-6d5d-4d56-b068-ee96eccf5fb7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfbb4d6809064f9cb9e06626ed7d6814', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '540a32ef88764e8c8d9201ae22f6ddaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:32:10.356199 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1330bbb4-6d5d-4d56-b068-ee96eccf5fb7 service neutron] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:32:10.356479 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1330bbb4-6d5d-4d56-b068-ee96eccf5fb7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:32:10.356767 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1330bbb4-6d5d-4d56-b068-ee96eccf5fb7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:32:10.366107 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1330bbb4-6d5d-4d56-b068-ee96eccf5fb7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:32:10.367703 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1330bbb4-6d5d-4d56-b068-ee96eccf5fb7 service neutron] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:32:10.378047 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1390/2810] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 12:32:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 12:32:10.655743 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a073e178-f7f3-4e92-ac27-ffb156c21fa7 None None] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:32:10.657276 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-a073e178-f7f3-4e92-ac27-ffb156c21fa7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:32:10.670503 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a073e178-f7f3-4e92-ac27-ffb156c21fa7 None None] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:32:10.675567 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a753ba6-b34c-4c98-8c43-8dd9e6254030 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:32:10.676157 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a753ba6-b34c-4c98-8c43-8dd9e6254030 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:32:10.676437 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a753ba6-b34c-4c98-8c43-8dd9e6254030 admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:32:10.676778 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a753ba6-b34c-4c98-8c43-8dd9e6254030 admin admin] PATH_INFO: `/v3/projects/cdcc8573ba58467f90a5ae3d41e2b009` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:32:10.677971 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a753ba6-b34c-4c98-8c43-8dd9e6254030 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cdcc8573ba58467f90a5ae3d41e2b009)` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:32:10.679803 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a753ba6-b34c-4c98-8c43-8dd9e6254030 admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:32:10.700723 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:32:10.740805 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-6a753ba6-b34c-4c98-8c43-8dd9e6254030 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cdcc8573ba58467f90a5ae3d41e2b009', 'request_id': 'req-6a753ba6-b34c-4c98-8c43-8dd9e6254030'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:32:10.748591 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-6a753ba6-b34c-4c98-8c43-8dd9e6254030 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cdcc8573ba58467f90a5ae3d41e2b009', 'request_id': 'req-6a753ba6-b34c-4c98-8c43-8dd9e6254030'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:32:10.754380 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-6a753ba6-b34c-4c98-8c43-8dd9e6254030 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cdcc8573ba58467f90a5ae3d41e2b009', 'request_id': 'req-6a753ba6-b34c-4c98-8c43-8dd9e6254030'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:32:10.766565 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-6a753ba6-b34c-4c98-8c43-8dd9e6254030 admin admin] The token cache is being invalidate because project cdcc8573ba58467f90a5ae3d41e2b009 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81449) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 12:32:10.766847 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-6a753ba6-b34c-4c98-8c43-8dd9e6254030 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:32:10.767578 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-6a753ba6-b34c-4c98-8c43-8dd9e6254030 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:32:10.769123 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1421/2811] 10.222.0.15 () {58 vars in 1233 bytes} [Thu Sep 15 12:32:10 2022] DELETE /identity/v3/projects/cdcc8573ba58467f90a5ae3d41e2b009 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:32:10.774612 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971a998f-4a01-473f-85c8-b231ae77fdb1 None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:32:10.774902 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971a998f-4a01-473f-85c8-b231ae77fdb1 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:32:10.775175 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-971a998f-4a01-473f-85c8-b231ae77fdb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:32:10.823390 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-971a998f-4a01-473f-85c8-b231ae77fdb1 None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:32:10.824607 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-971a998f-4a01-473f-85c8-b231ae77fdb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:32:10.885319 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1391/2812] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 12:32:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:32:10.893122 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a64e86f-5df6-450c-909c-ea81cc5b92ec None None] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:32:10.893484 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a64e86f-5df6-450c-909c-ea81cc5b92ec None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:32:10.893806 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a64e86f-5df6-450c-909c-ea81cc5b92ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:32:10.914011 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.auth.core [None req-7a64e86f-5df6-450c-909c-ea81cc5b92ec None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81449) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:32:10.915131 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-7a64e86f-5df6-450c-909c-ea81cc5b92ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:32:10.922868 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1422/2813] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 12:32:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:32:10.927236 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971a998f-4a01-473f-85c8-b231ae77fdb1 None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:32:10.937656 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-971a998f-4a01-473f-85c8-b231ae77fdb1 None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:32:10.942865 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296715bc-6174-4564-8436-9eba2113eaad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:32:10.943476 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296715bc-6174-4564-8436-9eba2113eaad admin admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:32:10.943757 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296715bc-6174-4564-8436-9eba2113eaad admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:32:10.944031 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296715bc-6174-4564-8436-9eba2113eaad admin admin] PATH_INFO: `/v3/domains` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:32:10.944680 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-296715bc-6174-4564-8436-9eba2113eaad admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:32:10.945016 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-296715bc-6174-4564-8436-9eba2113eaad admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:32:10.946613 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-296715bc-6174-4564-8436-9eba2113eaad admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:32:10.962338 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1392/2814] 10.222.0.15 () {58 vars in 1153 bytes} [Thu Sep 15 12:32:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:32:10.966737 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a64e86f-5df6-450c-909c-ea81cc5b92ec None None] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:32:10.978129 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a64e86f-5df6-450c-909c-ea81cc5b92ec None None] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:32:10.982755 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec04ae1-614b-439a-8774-5d22ec217e06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:32:10.983357 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec04ae1-614b-439a-8774-5d22ec217e06 admin admin] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:32:10.983634 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec04ae1-614b-439a-8774-5d22ec217e06 admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:32:10.983905 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec04ae1-614b-439a-8774-5d22ec217e06 admin admin] PATH_INFO: `/v3/projects` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:32:10.984530 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec04ae1-614b-439a-8774-5d22ec217e06 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:32:10.986452 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec04ae1-614b-439a-8774-5d22ec217e06 admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:32:11.014223 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1423/2815] 10.222.0.15 () {60 vars in 1153 bytes} [Thu Sep 15 12:32:10 2022] POST /identity/v3/projects => generated 397 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 12:32:11.018440 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296715bc-6174-4564-8436-9eba2113eaad admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:32:11.028835 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296715bc-6174-4564-8436-9eba2113eaad admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:32:11.084609 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d88877-7603-484a-bcfc-9f54c9d7c6e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:32:11.085304 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d88877-7603-484a-bcfc-9f54c9d7c6e2 admin admin] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:32:11.085509 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d88877-7603-484a-bcfc-9f54c9d7c6e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:32:11.085773 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d88877-7603-484a-bcfc-9f54c9d7c6e2 admin admin] PATH_INFO: `/v3/users` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:32:11.086409 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d88877-7603-484a-bcfc-9f54c9d7c6e2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:32:11.088338 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d88877-7603-484a-bcfc-9f54c9d7c6e2 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:32:11.113649 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1393/2816] 10.222.0.15 () {60 vars in 1144 bytes} [Thu Sep 15 12:32:11 2022] POST /identity/v3/users => generated 299 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 12:32:11.118172 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec04ae1-614b-439a-8774-5d22ec217e06 admin admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:32:11.129107 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec04ae1-614b-439a-8774-5d22ec217e06 admin admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:32:11.134721 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b0b62c-4844-46a3-93ed-7f80532b697e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:32:11.135320 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b0b62c-4844-46a3-93ed-7f80532b697e admin admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:32:11.135721 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b0b62c-4844-46a3-93ed-7f80532b697e admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:32:11.136003 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b0b62c-4844-46a3-93ed-7f80532b697e admin admin] PATH_INFO: `/v3/roles` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:32:11.136653 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b0b62c-4844-46a3-93ed-7f80532b697e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:32:11.138287 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b0b62c-4844-46a3-93ed-7f80532b697e admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:32:11.148958 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1424/2817] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 12:32:11 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 12:32:11.153147 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d88877-7603-484a-bcfc-9f54c9d7c6e2 admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:32:11.163056 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d88877-7603-484a-bcfc-9f54c9d7c6e2 admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:32:11.167995 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e01e31d-0eed-4947-ab38-e6a37a098a88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:32:11.168531 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e01e31d-0eed-4947-ab38-e6a37a098a88 admin admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:32:11.168854 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e01e31d-0eed-4947-ab38-e6a37a098a88 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:32:11.169129 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e01e31d-0eed-4947-ab38-e6a37a098a88 admin admin] PATH_INFO: `/v3/roles` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:32:11.169959 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e01e31d-0eed-4947-ab38-e6a37a098a88 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:32:11.171759 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e01e31d-0eed-4947-ab38-e6a37a098a88 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:32:11.183428 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1394/2818] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 12:32:11 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 12:32:11.187742 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b0b62c-4844-46a3-93ed-7f80532b697e admin admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:32:11.198966 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b0b62c-4844-46a3-93ed-7f80532b697e admin admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:32:11.204293 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7eca452-dae0-4e6e-8023-e67587d140c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:32:11.205000 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7eca452-dae0-4e6e-8023-e67587d140c8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:32:11.205282 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7eca452-dae0-4e6e-8023-e67587d140c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:32:11.205556 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7eca452-dae0-4e6e-8023-e67587d140c8 admin admin] PATH_INFO: `/v3/projects/55c7e09ba17142acaafd0526f740bf40/users/228d9092b5ca4c2cafebcbc5a4834950/roles/0679d6628a25438d90dc4bdc0401eda2` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:32:11.220616 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7eca452-dae0-4e6e-8023-e67587d140c8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=55c7e09ba17142acaafd0526f740bf40, user_id=228d9092b5ca4c2cafebcbc5a4834950, role_id=0679d6628a25438d90dc4bdc0401eda2)` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:32:11.223416 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7eca452-dae0-4e6e-8023-e67587d140c8 admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:32:11.237328 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1425/2819] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 15 12:32:11 2022] PUT /identity/v3/projects/55c7e09ba17142acaafd0526f740bf40/users/228d9092b5ca4c2cafebcbc5a4834950/roles/0679d6628a25438d90dc4bdc0401eda2 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:32:11.247782 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d896492c-e25d-48da-9ff9-f7a054f08101 None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:32:11.248073 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d896492c-e25d-48da-9ff9-f7a054f08101 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:32:11.248352 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d896492c-e25d-48da-9ff9-f7a054f08101 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:32:11.269184 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:32:11.281345 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-d896492c-e25d-48da-9ff9-f7a054f08101 None None] MFA Rules not processed for user `228d9092b5ca4c2cafebcbc5a4834950`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:32:11.282555 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-d896492c-e25d-48da-9ff9-f7a054f08101 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:32:11.350605 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1395/2820] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 12:32:11 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:32:11.362036 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7eca452-dae0-4e6e-8023-e67587d140c8 admin admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:32:11.363706 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-c7eca452-dae0-4e6e-8023-e67587d140c8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:32:11.423795 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7eca452-dae0-4e6e-8023-e67587d140c8 admin admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:32:11.427956 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69881912-d41a-488e-8c7b-28a37dea9e68 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8191d3ccb874529b32f8d0026db629d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '540a32ef88764e8c8d9201ae22f6ddaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:32:11.428649 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69881912-d41a-488e-8c7b-28a37dea9e68 service cinder] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:32:11.428974 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69881912-d41a-488e-8c7b-28a37dea9e68 service cinder] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:32:11.429248 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69881912-d41a-488e-8c7b-28a37dea9e68 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:32:11.438488 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69881912-d41a-488e-8c7b-28a37dea9e68 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:32:11.439922 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69881912-d41a-488e-8c7b-28a37dea9e68 service cinder] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:32:11.448817 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1426/2821] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 12:32:11 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 12:32:19.539601 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d896492c-e25d-48da-9ff9-f7a054f08101 None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:32:19.611505 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d896492c-e25d-48da-9ff9-f7a054f08101 None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:32:19.616176 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55509dc2-957d-417f-a5cc-6a40c8ef8b83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:32:19.616853 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55509dc2-957d-417f-a5cc-6a40c8ef8b83 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:32:19.617130 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55509dc2-957d-417f-a5cc-6a40c8ef8b83 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:32:19.617403 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55509dc2-957d-417f-a5cc-6a40c8ef8b83 admin admin] PATH_INFO: `/v3/users/228d9092b5ca4c2cafebcbc5a4834950` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:32:19.618565 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55509dc2-957d-417f-a5cc-6a40c8ef8b83 admin admin] RBAC: Authorizing `identity:delete_user(user_id=228d9092b5ca4c2cafebcbc5a4834950)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:32:19.622531 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55509dc2-957d-417f-a5cc-6a40c8ef8b83 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:32:19.670284 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-55509dc2-957d-417f-a5cc-6a40c8ef8b83 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '228d9092b5ca4c2cafebcbc5a4834950'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:32:19.682745 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-55509dc2-957d-417f-a5cc-6a40c8ef8b83 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '228d9092b5ca4c2cafebcbc5a4834950'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:32:19.691452 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-55509dc2-957d-417f-a5cc-6a40c8ef8b83 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '228d9092b5ca4c2cafebcbc5a4834950'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:32:19.692891 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-55509dc2-957d-417f-a5cc-6a40c8ef8b83 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '228d9092b5ca4c2cafebcbc5a4834950'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:32:19.700271 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-55509dc2-957d-417f-a5cc-6a40c8ef8b83 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '228d9092b5ca4c2cafebcbc5a4834950'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:32:19.706557 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1396/2822] 10.222.0.15 () {58 vars in 1224 bytes} [Thu Sep 15 12:32:19 2022] DELETE /identity/v3/users/228d9092b5ca4c2cafebcbc5a4834950 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:32:19.711496 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbd7185-ba38-408d-bc0d-ba9bc9ba765a None None] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:32:19.712265 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbd7185-ba38-408d-bc0d-ba9bc9ba765a None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:32:19.712265 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbd7185-ba38-408d-bc0d-ba9bc9ba765a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:32:19.732630 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.auth.core [None req-1cbd7185-ba38-408d-bc0d-ba9bc9ba765a None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81449) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:32:19.733949 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-1cbd7185-ba38-408d-bc0d-ba9bc9ba765a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:32:19.770075 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:32:19.799214 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1427/2823] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 12:32:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Sep 15 12:32:19.809910 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55509dc2-957d-417f-a5cc-6a40c8ef8b83 admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:32:19.811321 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-55509dc2-957d-417f-a5cc-6a40c8ef8b83 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:32:19.833870 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:32:19.874856 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55509dc2-957d-417f-a5cc-6a40c8ef8b83 admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:32:19.878507 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e289b88-aca8-467f-863b-2cc111acd7e8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfbb4d6809064f9cb9e06626ed7d6814', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '540a32ef88764e8c8d9201ae22f6ddaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:32:19.879154 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e289b88-aca8-467f-863b-2cc111acd7e8 service neutron] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:32:19.879434 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e289b88-aca8-467f-863b-2cc111acd7e8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:32:19.879703 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e289b88-aca8-467f-863b-2cc111acd7e8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:32:19.888931 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e289b88-aca8-467f-863b-2cc111acd7e8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:32:19.890354 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e289b88-aca8-467f-863b-2cc111acd7e8 service neutron] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:32:19.899979 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1397/2824] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 12:32:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 12:32:20.168716 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbd7185-ba38-408d-bc0d-ba9bc9ba765a None None] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:32:20.170134 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-1cbd7185-ba38-408d-bc0d-ba9bc9ba765a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:32:20.183137 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbd7185-ba38-408d-bc0d-ba9bc9ba765a None None] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:32:20.187613 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8387511-adc5-4be3-87d1-d93daf371738 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:32:20.188251 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8387511-adc5-4be3-87d1-d93daf371738 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:32:20.188524 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8387511-adc5-4be3-87d1-d93daf371738 admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:32:20.188820 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8387511-adc5-4be3-87d1-d93daf371738 admin admin] PATH_INFO: `/v3/projects/55c7e09ba17142acaafd0526f740bf40` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:32:20.189929 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8387511-adc5-4be3-87d1-d93daf371738 admin admin] RBAC: Authorizing `identity:delete_project(project_id=55c7e09ba17142acaafd0526f740bf40)` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:32:20.191850 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8387511-adc5-4be3-87d1-d93daf371738 admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:32:20.238529 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-f8387511-adc5-4be3-87d1-d93daf371738 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '55c7e09ba17142acaafd0526f740bf40', 'request_id': 'req-f8387511-adc5-4be3-87d1-d93daf371738'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:32:20.245793 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-f8387511-adc5-4be3-87d1-d93daf371738 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '55c7e09ba17142acaafd0526f740bf40', 'request_id': 'req-f8387511-adc5-4be3-87d1-d93daf371738'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:32:20.251370 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-f8387511-adc5-4be3-87d1-d93daf371738 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '55c7e09ba17142acaafd0526f740bf40', 'request_id': 'req-f8387511-adc5-4be3-87d1-d93daf371738'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:32:20.262279 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-f8387511-adc5-4be3-87d1-d93daf371738 admin admin] The token cache is being invalidate because project 55c7e09ba17142acaafd0526f740bf40 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81449) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 12:32:20.262596 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-f8387511-adc5-4be3-87d1-d93daf371738 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:32:20.263215 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-f8387511-adc5-4be3-87d1-d93daf371738 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:32:20.264665 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1428/2825] 10.222.0.15 () {58 vars in 1233 bytes} [Thu Sep 15 12:32:20 2022] DELETE /identity/v3/projects/55c7e09ba17142acaafd0526f740bf40 => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:32:20.454148 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e289b88-aca8-467f-863b-2cc111acd7e8 service neutron] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:32:20.455597 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-4e289b88-aca8-467f-863b-2cc111acd7e8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:32:20.557120 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e289b88-aca8-467f-863b-2cc111acd7e8 service neutron] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:32:20.633590 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4597099c-aaf8-4fd0-8ef6-218bd09eda99 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f64a058bd5440cb5268f228aa6e392', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '540a32ef88764e8c8d9201ae22f6ddaf', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:32:20.634649 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4597099c-aaf8-4fd0-8ef6-218bd09eda99 service placement] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:32:20.635041 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4597099c-aaf8-4fd0-8ef6-218bd09eda99 service placement] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:32:20.635564 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4597099c-aaf8-4fd0-8ef6-218bd09eda99 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:32:20.637888 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-4597099c-aaf8-4fd0-8ef6-218bd09eda99 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:32:20.651542 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4597099c-aaf8-4fd0-8ef6-218bd09eda99 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:32:20.653189 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4597099c-aaf8-4fd0-8ef6-218bd09eda99 service placement] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:32:20.663106 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1398/2826] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 12:32:20 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 209 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 12:32:30.259308 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=36,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:32:30.658736 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=85 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:32:56.242410 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0aabeb-cad9-4e97-ba96-712411eb8805 None None] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:32:56.242954 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0aabeb-cad9-4e97-ba96-712411eb8805 None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:32:56.242954 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0aabeb-cad9-4e97-ba96-712411eb8805 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:32:56.262424 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.auth.core [None req-8f0aabeb-cad9-4e97-ba96-712411eb8805 None None] MFA Rules not processed for user `f0a002a6c60e4666982e1cc74ed90573`. Rule list: `[]` (Enabled: `True`). {{(pid=81449) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:32:56.263618 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-8f0aabeb-cad9-4e97-ba96-712411eb8805 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:32:56.317387 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1429/2827] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 12:32:56 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:33:06.310792 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=25 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:33:51.471245 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4597099c-aaf8-4fd0-8ef6-218bd09eda99 service placement] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:33:51.472746 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-4597099c-aaf8-4fd0-8ef6-218bd09eda99 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:33:51.541094 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4597099c-aaf8-4fd0-8ef6-218bd09eda99 service placement] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:33:51.545562 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00174f57-8995-46bc-a5c5-9fe316566522 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:33:51.546192 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00174f57-8995-46bc-a5c5-9fe316566522 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:33:51.546475 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00174f57-8995-46bc-a5c5-9fe316566522 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:33:51.546744 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00174f57-8995-46bc-a5c5-9fe316566522 admin admin] PATH_INFO: `/v3/users/f0a002a6c60e4666982e1cc74ed90573` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:33:51.547867 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00174f57-8995-46bc-a5c5-9fe316566522 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f0a002a6c60e4666982e1cc74ed90573)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:33:51.550364 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00174f57-8995-46bc-a5c5-9fe316566522 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:33:51.595201 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-00174f57-8995-46bc-a5c5-9fe316566522 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f0a002a6c60e4666982e1cc74ed90573'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:33:51.607561 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-00174f57-8995-46bc-a5c5-9fe316566522 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f0a002a6c60e4666982e1cc74ed90573'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:33:51.614768 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-00174f57-8995-46bc-a5c5-9fe316566522 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f0a002a6c60e4666982e1cc74ed90573'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:33:51.615380 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-00174f57-8995-46bc-a5c5-9fe316566522 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f0a002a6c60e4666982e1cc74ed90573'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:33:51.622259 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-00174f57-8995-46bc-a5c5-9fe316566522 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f0a002a6c60e4666982e1cc74ed90573'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:33:51.628610 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1399/2828] 10.222.0.15 () {58 vars in 1224 bytes} [Thu Sep 15 12:33:51 2022] DELETE /identity/v3/users/f0a002a6c60e4666982e1cc74ed90573 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:33:51.801842 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0aabeb-cad9-4e97-ba96-712411eb8805 None None] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:33:51.818105 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0aabeb-cad9-4e97-ba96-712411eb8805 None None] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:33:51.873435 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01c6a8b2-1374-45a6-9d0e-6f8cdd54b19f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:33:51.874077 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c6a8b2-1374-45a6-9d0e-6f8cdd54b19f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:33:51.874356 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c6a8b2-1374-45a6-9d0e-6f8cdd54b19f admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:33:51.874625 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c6a8b2-1374-45a6-9d0e-6f8cdd54b19f admin admin] PATH_INFO: `/v3/projects/daddeef840a04bb8bc0d7b1f8cb37c7b` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:33:51.875725 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c6a8b2-1374-45a6-9d0e-6f8cdd54b19f admin admin] RBAC: Authorizing `identity:delete_project(project_id=daddeef840a04bb8bc0d7b1f8cb37c7b)` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:33:51.877650 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01c6a8b2-1374-45a6-9d0e-6f8cdd54b19f admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:33:51.926685 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-01c6a8b2-1374-45a6-9d0e-6f8cdd54b19f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'daddeef840a04bb8bc0d7b1f8cb37c7b', 'request_id': 'req-01c6a8b2-1374-45a6-9d0e-6f8cdd54b19f'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:33:51.933694 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-01c6a8b2-1374-45a6-9d0e-6f8cdd54b19f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'daddeef840a04bb8bc0d7b1f8cb37c7b', 'request_id': 'req-01c6a8b2-1374-45a6-9d0e-6f8cdd54b19f'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:33:51.939461 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-01c6a8b2-1374-45a6-9d0e-6f8cdd54b19f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'daddeef840a04bb8bc0d7b1f8cb37c7b', 'request_id': 'req-01c6a8b2-1374-45a6-9d0e-6f8cdd54b19f'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:33:51.950752 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-01c6a8b2-1374-45a6-9d0e-6f8cdd54b19f admin admin] The token cache is being invalidate because project daddeef840a04bb8bc0d7b1f8cb37c7b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81449) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 12:33:51.951034 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-01c6a8b2-1374-45a6-9d0e-6f8cdd54b19f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:33:51.951646 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-01c6a8b2-1374-45a6-9d0e-6f8cdd54b19f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:33:51.953128 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1430/2829] 10.222.0.15 () {58 vars in 1233 bytes} [Thu Sep 15 12:33:51 2022] DELETE /identity/v3/projects/daddeef840a04bb8bc0d7b1f8cb37c7b => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:33:51.958941 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52aaa1d8-5276-4eec-8190-b9ee0f97ceb7 None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:33:51.959298 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52aaa1d8-5276-4eec-8190-b9ee0f97ceb7 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:33:51.959644 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52aaa1d8-5276-4eec-8190-b9ee0f97ceb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:33:51.979310 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-52aaa1d8-5276-4eec-8190-b9ee0f97ceb7 None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:33:51.980503 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-52aaa1d8-5276-4eec-8190-b9ee0f97ceb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:33:52.038108 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1400/2830] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 12:33:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:33:52.046859 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79dd0b7-21e6-4770-a591-6654f859510b None None] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:33:52.047088 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79dd0b7-21e6-4770-a591-6654f859510b None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:33:52.047367 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79dd0b7-21e6-4770-a591-6654f859510b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:33:52.066910 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.auth.core [None req-f79dd0b7-21e6-4770-a591-6654f859510b None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81449) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:33:52.068105 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-f79dd0b7-21e6-4770-a591-6654f859510b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:33:52.075982 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1431/2831] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 12:33:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:33:52.080387 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52aaa1d8-5276-4eec-8190-b9ee0f97ceb7 None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:33:52.090858 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52aaa1d8-5276-4eec-8190-b9ee0f97ceb7 None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:33:52.095705 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40703181-2225-4ef3-b914-d3b10072ab5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:33:52.096306 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40703181-2225-4ef3-b914-d3b10072ab5e admin admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:33:52.096580 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40703181-2225-4ef3-b914-d3b10072ab5e admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:33:52.096883 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40703181-2225-4ef3-b914-d3b10072ab5e admin admin] PATH_INFO: `/v3/domains` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:33:52.097524 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40703181-2225-4ef3-b914-d3b10072ab5e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:33:52.097817 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40703181-2225-4ef3-b914-d3b10072ab5e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:33:52.099390 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40703181-2225-4ef3-b914-d3b10072ab5e admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:33:52.111492 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1401/2832] 10.222.0.15 () {58 vars in 1153 bytes} [Thu Sep 15 12:33:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:33:52.115937 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79dd0b7-21e6-4770-a591-6654f859510b None None] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:33:52.125111 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f79dd0b7-21e6-4770-a591-6654f859510b None None] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:33:52.130237 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0f25a7-c395-4c58-8f7e-38e9552e52e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:33:52.130829 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0f25a7-c395-4c58-8f7e-38e9552e52e7 admin admin] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:33:52.131106 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0f25a7-c395-4c58-8f7e-38e9552e52e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:33:52.131378 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0f25a7-c395-4c58-8f7e-38e9552e52e7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:33:52.131996 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0f25a7-c395-4c58-8f7e-38e9552e52e7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:33:52.133922 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0f25a7-c395-4c58-8f7e-38e9552e52e7 admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:33:52.160465 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1432/2833] 10.222.0.15 () {60 vars in 1153 bytes} [Thu Sep 15 12:33:52 2022] POST /identity/v3/projects => generated 385 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 12:33:52.164619 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40703181-2225-4ef3-b914-d3b10072ab5e admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:33:52.174487 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40703181-2225-4ef3-b914-d3b10072ab5e admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:33:52.178513 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:33:52.232546 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f2d13f-ccde-4a75-9901-57381a647fca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:33:52.233179 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f2d13f-ccde-4a75-9901-57381a647fca admin admin] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:33:52.233463 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f2d13f-ccde-4a75-9901-57381a647fca admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:33:52.233734 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f2d13f-ccde-4a75-9901-57381a647fca admin admin] PATH_INFO: `/v3/users` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:33:52.234354 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f2d13f-ccde-4a75-9901-57381a647fca admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:33:52.236250 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f2d13f-ccde-4a75-9901-57381a647fca admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:33:52.257575 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1402/2834] 10.222.0.15 () {60 vars in 1144 bytes} [Thu Sep 15 12:33:52 2022] POST /identity/v3/users => generated 293 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 12:33:52.262029 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0f25a7-c395-4c58-8f7e-38e9552e52e7 admin admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:33:52.271979 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0f25a7-c395-4c58-8f7e-38e9552e52e7 admin admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:33:52.276462 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ddcee0f-0ac6-4afa-96fd-8bb0cf2321f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:33:52.277087 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddcee0f-0ac6-4afa-96fd-8bb0cf2321f1 admin admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:33:52.277363 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddcee0f-0ac6-4afa-96fd-8bb0cf2321f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:33:52.277651 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddcee0f-0ac6-4afa-96fd-8bb0cf2321f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:33:52.278265 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ddcee0f-0ac6-4afa-96fd-8bb0cf2321f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:33:52.279828 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ddcee0f-0ac6-4afa-96fd-8bb0cf2321f1 admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:33:52.289430 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1433/2835] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 12:33:52 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 15 12:33:52.293157 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f2d13f-ccde-4a75-9901-57381a647fca admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:33:52.303363 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f2d13f-ccde-4a75-9901-57381a647fca admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:33:52.307832 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8eeea5-2da2-45bc-9804-1e3de1bef9c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:33:52.308425 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8eeea5-2da2-45bc-9804-1e3de1bef9c3 admin admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:33:52.308704 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8eeea5-2da2-45bc-9804-1e3de1bef9c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:33:52.309014 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8eeea5-2da2-45bc-9804-1e3de1bef9c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:33:52.309647 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8eeea5-2da2-45bc-9804-1e3de1bef9c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:33:52.311219 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8eeea5-2da2-45bc-9804-1e3de1bef9c3 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:33:52.322544 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1403/2836] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 12:33:52 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 12:33:52.326597 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ddcee0f-0ac6-4afa-96fd-8bb0cf2321f1 admin admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:33:52.336679 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ddcee0f-0ac6-4afa-96fd-8bb0cf2321f1 admin admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:33:52.341238 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab43818c-2b87-43de-80ed-2e431dcbf980 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:33:52.341859 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab43818c-2b87-43de-80ed-2e431dcbf980 admin admin] REQUEST_METHOD: `PUT` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:33:52.342133 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab43818c-2b87-43de-80ed-2e431dcbf980 admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:33:52.342399 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab43818c-2b87-43de-80ed-2e431dcbf980 admin admin] PATH_INFO: `/v3/projects/84b799b1cfab4245bf9fbfce0faf036a/users/08bd9f54467341afacba19215b78bf59/roles/0679d6628a25438d90dc4bdc0401eda2` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:33:52.356394 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab43818c-2b87-43de-80ed-2e431dcbf980 admin admin] RBAC: Authorizing `identity:create_grant(project_id=84b799b1cfab4245bf9fbfce0faf036a, user_id=08bd9f54467341afacba19215b78bf59, role_id=0679d6628a25438d90dc4bdc0401eda2)` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:33:52.359033 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab43818c-2b87-43de-80ed-2e431dcbf980 admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:33:52.371756 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1434/2837] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 15 12:33:52 2022] PUT /identity/v3/projects/84b799b1cfab4245bf9fbfce0faf036a/users/08bd9f54467341afacba19215b78bf59/roles/0679d6628a25438d90dc4bdc0401eda2 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:33:52.376840 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f4a537-911f-40f2-af6a-977363da169f None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:33:52.377151 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f4a537-911f-40f2-af6a-977363da169f None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:33:52.377427 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f4a537-911f-40f2-af6a-977363da169f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:33:52.396557 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-a7f4a537-911f-40f2-af6a-977363da169f None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:33:52.397810 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-a7f4a537-911f-40f2-af6a-977363da169f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:33:52.459194 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1404/2838] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 12:33:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:33:52.469090 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab43818c-2b87-43de-80ed-2e431dcbf980 admin admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:33:52.470796 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-ab43818c-2b87-43de-80ed-2e431dcbf980 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:33:52.500929 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:33:52.549734 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab43818c-2b87-43de-80ed-2e431dcbf980 admin admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:33:52.554068 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ec8c3a-226d-40e1-a1b0-1f6769260d38 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfbb4d6809064f9cb9e06626ed7d6814', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '540a32ef88764e8c8d9201ae22f6ddaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:33:52.554812 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ec8c3a-226d-40e1-a1b0-1f6769260d38 service neutron] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:33:52.555097 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ec8c3a-226d-40e1-a1b0-1f6769260d38 service neutron] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:33:52.555367 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ec8c3a-226d-40e1-a1b0-1f6769260d38 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:33:52.564012 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ec8c3a-226d-40e1-a1b0-1f6769260d38 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:33:52.565483 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ec8c3a-226d-40e1-a1b0-1f6769260d38 service neutron] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:33:52.574594 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1435/2839] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 12:33:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 12:33:56.401136 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422b64d3-3721-4a14-8be1-83b2312354cf None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:33:56.401731 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422b64d3-3721-4a14-8be1-83b2312354cf None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:33:56.401836 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422b64d3-3721-4a14-8be1-83b2312354cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:33:56.431038 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-422b64d3-3721-4a14-8be1-83b2312354cf None None] MFA Rules not processed for user `08bd9f54467341afacba19215b78bf59`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:33:56.432253 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-422b64d3-3721-4a14-8be1-83b2312354cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:33:56.500378 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1405/2840] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 12:33:56 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:33:56.507883 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ec8c3a-226d-40e1-a1b0-1f6769260d38 service neutron] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:33:56.528626 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ec8c3a-226d-40e1-a1b0-1f6769260d38 service neutron] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:33:56.545803 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2868cc8-309f-4fee-a9de-05e0494938ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:33:56.546444 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2868cc8-309f-4fee-a9de-05e0494938ec admin admin] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:33:56.546723 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2868cc8-309f-4fee-a9de-05e0494938ec admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:33:56.546993 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2868cc8-309f-4fee-a9de-05e0494938ec admin admin] PATH_INFO: `/v3/projects` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:33:56.547625 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2868cc8-309f-4fee-a9de-05e0494938ec admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:33:56.549584 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2868cc8-309f-4fee-a9de-05e0494938ec admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:33:56.584638 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1436/2841] 10.222.0.15 () {60 vars in 1153 bytes} [Thu Sep 15 12:33:56 2022] POST /identity/v3/projects => generated 385 bytes in 77 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 12:33:56.588636 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422b64d3-3721-4a14-8be1-83b2312354cf None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:33:56.600313 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422b64d3-3721-4a14-8be1-83b2312354cf None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:33:56.612510 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:33:56.663410 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d982472c-b938-40bc-bc80-99c9972b2acc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:33:56.664027 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d982472c-b938-40bc-bc80-99c9972b2acc admin admin] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:33:56.664305 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d982472c-b938-40bc-bc80-99c9972b2acc admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:33:56.664572 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d982472c-b938-40bc-bc80-99c9972b2acc admin admin] PATH_INFO: `/v3/users` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:33:56.665260 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d982472c-b938-40bc-bc80-99c9972b2acc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:33:56.667166 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d982472c-b938-40bc-bc80-99c9972b2acc admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:33:56.695143 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1406/2842] 10.222.0.15 () {60 vars in 1144 bytes} [Thu Sep 15 12:33:56 2022] POST /identity/v3/users => generated 299 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 15 12:33:56.696851 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2868cc8-309f-4fee-a9de-05e0494938ec admin admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:33:56.709564 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2868cc8-309f-4fee-a9de-05e0494938ec admin admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:33:56.716283 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456f14b1-24ff-4e66-9fda-d032f1dacd2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:33:56.716980 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456f14b1-24ff-4e66-9fda-d032f1dacd2f admin admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:33:56.717258 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456f14b1-24ff-4e66-9fda-d032f1dacd2f admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:33:56.717729 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456f14b1-24ff-4e66-9fda-d032f1dacd2f admin admin] PATH_INFO: `/v3/roles` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:33:56.718501 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456f14b1-24ff-4e66-9fda-d032f1dacd2f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:33:56.719975 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456f14b1-24ff-4e66-9fda-d032f1dacd2f admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:33:56.731114 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1437/2843] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 12:33:56 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 12:33:56.735198 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d982472c-b938-40bc-bc80-99c9972b2acc admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:33:56.746115 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d982472c-b938-40bc-bc80-99c9972b2acc admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:33:56.750870 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae3f5d9-2773-44f4-ab37-fe1da9bf9f30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:33:56.751556 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae3f5d9-2773-44f4-ab37-fe1da9bf9f30 admin admin] REQUEST_METHOD: `PUT` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:33:56.751843 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae3f5d9-2773-44f4-ab37-fe1da9bf9f30 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:33:56.752120 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae3f5d9-2773-44f4-ab37-fe1da9bf9f30 admin admin] PATH_INFO: `/v3/projects/6e2586884a2a4eebb15dae6c061ac277/users/81c5a3f4327b4301b47d9d0623280922/roles/9c769b5c82884766b2c510b1f0f1364c` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:33:56.770521 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae3f5d9-2773-44f4-ab37-fe1da9bf9f30 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6e2586884a2a4eebb15dae6c061ac277, user_id=81c5a3f4327b4301b47d9d0623280922, role_id=9c769b5c82884766b2c510b1f0f1364c)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:33:56.773406 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae3f5d9-2773-44f4-ab37-fe1da9bf9f30 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:33:56.788566 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1407/2844] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 15 12:33:56 2022] PUT /identity/v3/projects/6e2586884a2a4eebb15dae6c061ac277/users/81c5a3f4327b4301b47d9d0623280922/roles/9c769b5c82884766b2c510b1f0f1364c => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:33:56.792572 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456f14b1-24ff-4e66-9fda-d032f1dacd2f admin admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:33:56.803089 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456f14b1-24ff-4e66-9fda-d032f1dacd2f admin admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:33:56.870053 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2385eef3-24f7-46ce-9f88-fb55fdb93efa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:33:56.870404 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2385eef3-24f7-46ce-9f88-fb55fdb93efa admin admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:33:56.870687 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2385eef3-24f7-46ce-9f88-fb55fdb93efa admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:33:56.870960 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2385eef3-24f7-46ce-9f88-fb55fdb93efa admin admin] PATH_INFO: `/v3/roles` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:33:56.871631 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2385eef3-24f7-46ce-9f88-fb55fdb93efa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:33:56.873275 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2385eef3-24f7-46ce-9f88-fb55fdb93efa admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:33:56.884580 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1438/2845] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 12:33:56 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 12:33:56.888746 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae3f5d9-2773-44f4-ab37-fe1da9bf9f30 admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:33:56.899479 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae3f5d9-2773-44f4-ab37-fe1da9bf9f30 admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:33:56.904149 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6807b637-a984-4859-aa83-a58b8b61da52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:33:56.904854 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6807b637-a984-4859-aa83-a58b8b61da52 admin admin] REQUEST_METHOD: `PUT` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:33:56.905135 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6807b637-a984-4859-aa83-a58b8b61da52 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:33:56.905405 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6807b637-a984-4859-aa83-a58b8b61da52 admin admin] PATH_INFO: `/v3/projects/6e2586884a2a4eebb15dae6c061ac277/users/81c5a3f4327b4301b47d9d0623280922/roles/9c769b5c82884766b2c510b1f0f1364c` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:33:56.907297 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6807b637-a984-4859-aa83-a58b8b61da52 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6e2586884a2a4eebb15dae6c061ac277, user_id=81c5a3f4327b4301b47d9d0623280922, role_id=9c769b5c82884766b2c510b1f0f1364c)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:33:56.910002 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6807b637-a984-4859-aa83-a58b8b61da52 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:33:56.921145 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1408/2846] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 15 12:33:56 2022] PUT /identity/v3/projects/6e2586884a2a4eebb15dae6c061ac277/users/81c5a3f4327b4301b47d9d0623280922/roles/9c769b5c82884766b2c510b1f0f1364c => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:34:01.371912 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3801fb-bad5-4d7a-9a1f-bd79d763b10f None None] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:34:01.372398 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3801fb-bad5-4d7a-9a1f-bd79d763b10f None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:34:01.372516 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3801fb-bad5-4d7a-9a1f-bd79d763b10f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:34:01.411398 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.auth.core [None req-5b3801fb-bad5-4d7a-9a1f-bd79d763b10f None None] MFA Rules not processed for user `81c5a3f4327b4301b47d9d0623280922`. Rule list: `[]` (Enabled: `True`). {{(pid=81449) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:34:01.412682 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-5b3801fb-bad5-4d7a-9a1f-bd79d763b10f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:34:01.478779 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:34:01.489653 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1439/2847] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 12:34:01 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:34:01.498014 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6807b637-a984-4859-aa83-a58b8b61da52 admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:34:01.499811 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-6807b637-a984-4859-aa83-a58b8b61da52 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:34:01.559588 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6807b637-a984-4859-aa83-a58b8b61da52 admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:34:01.563222 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba120de-028c-492b-bf01-2dfe72b29ba9 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3aed8f78903f44cda9e5254aa46cf2b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '540a32ef88764e8c8d9201ae22f6ddaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:34:01.563854 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba120de-028c-492b-bf01-2dfe72b29ba9 service glance] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:34:01.564127 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba120de-028c-492b-bf01-2dfe72b29ba9 service glance] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:34:01.564392 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba120de-028c-492b-bf01-2dfe72b29ba9 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:34:01.625623 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba120de-028c-492b-bf01-2dfe72b29ba9 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:34:01.626911 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba120de-028c-492b-bf01-2dfe72b29ba9 service glance] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:34:01.637811 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1409/2848] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 12:34:01 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 12:34:01.790941 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07e9e4f-c3b3-492f-b9ac-b1309ce17a2c None None] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:34:01.791225 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07e9e4f-c3b3-492f-b9ac-b1309ce17a2c None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:34:01.791495 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07e9e4f-c3b3-492f-b9ac-b1309ce17a2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:34:01.821542 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.auth.core [None req-d07e9e4f-c3b3-492f-b9ac-b1309ce17a2c None None] MFA Rules not processed for user `527f4dd6031546c785bc6360dacb60af`. Rule list: `[]` (Enabled: `True`). {{(pid=81449) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:34:01.822736 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-d07e9e4f-c3b3-492f-b9ac-b1309ce17a2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:34:01.890617 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1440/2849] 10.222.0.15 () {58 vars in 1009 bytes} [Thu Sep 15 12:34:01 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:34:01.939969 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba120de-028c-492b-bf01-2dfe72b29ba9 service glance] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:34:01.941472 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-dba120de-028c-492b-bf01-2dfe72b29ba9 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:34:01.969048 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:34:02.015426 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba120de-028c-492b-bf01-2dfe72b29ba9 service glance] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:34:02.020987 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8ebfc5-8882-4415-9b10-2965e36e90e2 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa0d3dc5b7540d59fdd59bbf64a04b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '540a32ef88764e8c8d9201ae22f6ddaf', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:34:02.021633 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8ebfc5-8882-4415-9b10-2965e36e90e2 service swift] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:34:02.021916 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8ebfc5-8882-4415-9b10-2965e36e90e2 service swift] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:34:02.022190 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8ebfc5-8882-4415-9b10-2965e36e90e2 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:34:02.030538 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8ebfc5-8882-4415-9b10-2965e36e90e2 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:34:02.032047 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8ebfc5-8882-4415-9b10-2965e36e90e2 service swift] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:34:02.041237 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1410/2850] 10.222.0.15 () {60 vars in 1384 bytes} [Thu Sep 15 12:34:01 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 15 12:34:10.637751 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a85338-b6da-46cf-bdb9-0f8376c1804a None None] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:34:10.638304 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a85338-b6da-46cf-bdb9-0f8376c1804a None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:34:10.638416 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a85338-b6da-46cf-bdb9-0f8376c1804a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:34:10.657292 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.auth.core [None req-d9a85338-b6da-46cf-bdb9-0f8376c1804a None None] MFA Rules not processed for user `08bd9f54467341afacba19215b78bf59`. Rule list: `[]` (Enabled: `True`). {{(pid=81449) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:34:10.658993 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-d9a85338-b6da-46cf-bdb9-0f8376c1804a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:34:10.665734 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1441/2851] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 12:34:10 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:34:10.705951 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8ebfc5-8882-4415-9b10-2965e36e90e2 service swift] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:34:10.715273 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8ebfc5-8882-4415-9b10-2965e36e90e2 service swift] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:34:10.718880 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949f4af3-ab8d-49e8-aef1-665c15c7d529 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3aed8f78903f44cda9e5254aa46cf2b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '540a32ef88764e8c8d9201ae22f6ddaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:34:10.719514 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949f4af3-ab8d-49e8-aef1-665c15c7d529 service glance] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:34:10.719791 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949f4af3-ab8d-49e8-aef1-665c15c7d529 service glance] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:34:10.720060 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949f4af3-ab8d-49e8-aef1-665c15c7d529 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:34:10.722602 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-949f4af3-ab8d-49e8-aef1-665c15c7d529 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:34:10.805086 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949f4af3-ab8d-49e8-aef1-665c15c7d529 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:34:10.806517 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949f4af3-ab8d-49e8-aef1-665c15c7d529 service glance] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:34:10.827820 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1411/2852] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 12:34:10 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 12:34:12.643286 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77f6e0f-032f-488c-b26b-098c5f32a1af None None] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:34:12.643849 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77f6e0f-032f-488c-b26b-098c5f32a1af None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:34:12.643849 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77f6e0f-032f-488c-b26b-098c5f32a1af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:34:12.662830 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.auth.core [None req-c77f6e0f-032f-488c-b26b-098c5f32a1af None None] MFA Rules not processed for user `527f4dd6031546c785bc6360dacb60af`. Rule list: `[]` (Enabled: `True`). {{(pid=81449) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:34:12.664044 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-c77f6e0f-032f-488c-b26b-098c5f32a1af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:34:12.671209 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1442/2853] 10.222.0.15 () {58 vars in 1009 bytes} [Thu Sep 15 12:34:12 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:34:12.683689 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949f4af3-ab8d-49e8-aef1-665c15c7d529 service glance] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:34:12.696781 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949f4af3-ab8d-49e8-aef1-665c15c7d529 service glance] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:34:12.702003 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74eab0c-dbfc-405b-ba58-fb16306afd55 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa0d3dc5b7540d59fdd59bbf64a04b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '540a32ef88764e8c8d9201ae22f6ddaf', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:34:12.702829 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74eab0c-dbfc-405b-ba58-fb16306afd55 service swift] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:34:12.703035 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74eab0c-dbfc-405b-ba58-fb16306afd55 service swift] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:34:12.703372 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74eab0c-dbfc-405b-ba58-fb16306afd55 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:34:12.712776 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c74eab0c-dbfc-405b-ba58-fb16306afd55 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:34:12.714359 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c74eab0c-dbfc-405b-ba58-fb16306afd55 service swift] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:34:12.723013 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1412/2854] 10.222.0.15 () {60 vars in 1384 bytes} [Thu Sep 15 12:34:12 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Sep 15 12:34:13.581787 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77f6e0f-032f-488c-b26b-098c5f32a1af None None] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:34:13.643285 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77f6e0f-032f-488c-b26b-098c5f32a1af None None] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:34:13.648351 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6924a808-d217-48ac-8512-88f4f336d587 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfbb4d6809064f9cb9e06626ed7d6814', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '540a32ef88764e8c8d9201ae22f6ddaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:34:13.649270 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6924a808-d217-48ac-8512-88f4f336d587 service neutron] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:34:13.649825 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6924a808-d217-48ac-8512-88f4f336d587 service neutron] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:34:13.650316 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6924a808-d217-48ac-8512-88f4f336d587 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:34:13.652517 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-6924a808-d217-48ac-8512-88f4f336d587 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:34:13.668040 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6924a808-d217-48ac-8512-88f4f336d587 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:34:13.669532 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6924a808-d217-48ac-8512-88f4f336d587 service neutron] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:34:13.681148 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1443/2855] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 12:34:13 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 12:34:22.719438 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=71 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:34:23.675979 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=65 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:34:37.386521 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74eab0c-dbfc-405b-ba58-fb16306afd55 service swift] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:34:37.396247 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74eab0c-dbfc-405b-ba58-fb16306afd55 service swift] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:34:37.399763 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc0700b-8917-43c5-96f1-90f2ff2b7d92 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfbb4d6809064f9cb9e06626ed7d6814', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '540a32ef88764e8c8d9201ae22f6ddaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:34:37.400355 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc0700b-8917-43c5-96f1-90f2ff2b7d92 service neutron] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:34:37.400633 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc0700b-8917-43c5-96f1-90f2ff2b7d92 service neutron] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:34:37.400946 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc0700b-8917-43c5-96f1-90f2ff2b7d92 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:34:37.410176 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc0700b-8917-43c5-96f1-90f2ff2b7d92 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:34:37.411576 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc0700b-8917-43c5-96f1-90f2ff2b7d92 service neutron] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:34:37.420825 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1413/2856] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 12:34:37 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 12:34:38.952745 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6924a808-d217-48ac-8512-88f4f336d587 service neutron] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:34:38.955259 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-6924a808-d217-48ac-8512-88f4f336d587 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:34:38.979749 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6924a808-d217-48ac-8512-88f4f336d587 service neutron] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:34:38.985832 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb494cf-e04c-4ff6-adc4-52307e0d40ef service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eface89e261840a2abb4206be26455c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '540a32ef88764e8c8d9201ae22f6ddaf', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:34:38.986601 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb494cf-e04c-4ff6-adc4-52307e0d40ef service nova] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:34:38.986877 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb494cf-e04c-4ff6-adc4-52307e0d40ef service nova] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:34:38.987158 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb494cf-e04c-4ff6-adc4-52307e0d40ef service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:34:38.989233 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-bcb494cf-e04c-4ff6-adc4-52307e0d40ef service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:34:39.002133 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb494cf-e04c-4ff6-adc4-52307e0d40ef service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:34:39.004001 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb494cf-e04c-4ff6-adc4-52307e0d40ef service nova] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:34:39.016056 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1444/2857] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 12:34:38 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 64 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 12:34:47.416693 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=6 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:34:49.011013 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=6 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:35:04.429373 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc0700b-8917-43c5-96f1-90f2ff2b7d92 service neutron] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:35:04.430847 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-cdc0700b-8917-43c5-96f1-90f2ff2b7d92 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:35:04.442538 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc0700b-8917-43c5-96f1-90f2ff2b7d92 service neutron] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:35:04.446120 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b158d9-74e2-4036-980d-de5317488085 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfbb4d6809064f9cb9e06626ed7d6814', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '540a32ef88764e8c8d9201ae22f6ddaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:35:04.446720 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b158d9-74e2-4036-980d-de5317488085 service neutron] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:35:04.446997 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b158d9-74e2-4036-980d-de5317488085 service neutron] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:35:04.447265 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b158d9-74e2-4036-980d-de5317488085 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:35:04.448997 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-62b158d9-74e2-4036-980d-de5317488085 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:35:04.458841 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b158d9-74e2-4036-980d-de5317488085 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:35:04.460254 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b158d9-74e2-4036-980d-de5317488085 service neutron] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:35:04.469039 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1414/2858] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 12:35:04 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 12:35:14.466004 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=6 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:35:56.673731 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb494cf-e04c-4ff6-adc4-52307e0d40ef service nova] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:35:56.688413 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb494cf-e04c-4ff6-adc4-52307e0d40ef service nova] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:35:56.694190 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc06668-cf36-46fa-9743-9fbb3fbe6152 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eface89e261840a2abb4206be26455c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '540a32ef88764e8c8d9201ae22f6ddaf', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:35:56.694901 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc06668-cf36-46fa-9743-9fbb3fbe6152 service nova] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:35:56.695258 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc06668-cf36-46fa-9743-9fbb3fbe6152 service nova] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:35:56.695591 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc06668-cf36-46fa-9743-9fbb3fbe6152 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:35:56.697445 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-2fc06668-cf36-46fa-9743-9fbb3fbe6152 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:35:56.711485 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc06668-cf36-46fa-9743-9fbb3fbe6152 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:35:56.713328 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc06668-cf36-46fa-9743-9fbb3fbe6152 service nova] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:35:56.733077 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1445/2859] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 12:35:56 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 12:35:57.821306 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b3ba0f-f323-4e12-a270-56164a07bc3d None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:35:57.821782 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b3ba0f-f323-4e12-a270-56164a07bc3d None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:35:57.821883 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b3ba0f-f323-4e12-a270-56164a07bc3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:35:57.871068 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-a4b3ba0f-f323-4e12-a270-56164a07bc3d None None] MFA Rules not processed for user `527f4dd6031546c785bc6360dacb60af`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:35:57.872272 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-a4b3ba0f-f323-4e12-a270-56164a07bc3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:35:57.888291 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1415/2860] 10.222.0.15 () {58 vars in 967 bytes} [Thu Sep 15 12:35:57 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 68 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:35:57.928238 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc06668-cf36-46fa-9743-9fbb3fbe6152 service nova] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:35:57.948163 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc06668-cf36-46fa-9743-9fbb3fbe6152 service nova] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:35:57.953034 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942cc6fe-737b-48d2-86e9-2d7b84740550 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'efa0d3dc5b7540d59fdd59bbf64a04b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '540a32ef88764e8c8d9201ae22f6ddaf', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:35:57.953631 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942cc6fe-737b-48d2-86e9-2d7b84740550 service swift] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:35:57.953907 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942cc6fe-737b-48d2-86e9-2d7b84740550 service swift] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:35:57.954176 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942cc6fe-737b-48d2-86e9-2d7b84740550 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:35:57.962444 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942cc6fe-737b-48d2-86e9-2d7b84740550 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:35:57.964006 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942cc6fe-737b-48d2-86e9-2d7b84740550 service swift] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:35:57.972577 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1446/2861] 10.222.0.15 () {60 vars in 1384 bytes} [Thu Sep 15 12:35:57 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Sep 15 12:36:01.299055 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b3ba0f-f323-4e12-a270-56164a07bc3d None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:36:01.300744 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-a4b3ba0f-f323-4e12-a270-56164a07bc3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:36:01.387274 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b3ba0f-f323-4e12-a270-56164a07bc3d None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:36:01.392269 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925aa6be-4500-4888-8a6c-d8c3e87d7fd4 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f64a058bd5440cb5268f228aa6e392', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '540a32ef88764e8c8d9201ae22f6ddaf', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:36:01.392937 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925aa6be-4500-4888-8a6c-d8c3e87d7fd4 service placement] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:36:01.393732 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925aa6be-4500-4888-8a6c-d8c3e87d7fd4 service placement] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:36:01.394002 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925aa6be-4500-4888-8a6c-d8c3e87d7fd4 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:36:01.395815 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-925aa6be-4500-4888-8a6c-d8c3e87d7fd4 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:36:01.407317 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-925aa6be-4500-4888-8a6c-d8c3e87d7fd4 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:36:01.410114 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-925aa6be-4500-4888-8a6c-d8c3e87d7fd4 service placement] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:36:01.423597 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1416/2862] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 12:36:01 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 12:36:04.360074 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942cc6fe-737b-48d2-86e9-2d7b84740550 service swift] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:36:04.371746 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942cc6fe-737b-48d2-86e9-2d7b84740550 service swift] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:36:04.376882 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f0afed-f095-4d9c-b77f-093466b68669 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f64a058bd5440cb5268f228aa6e392', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '540a32ef88764e8c8d9201ae22f6ddaf', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:36:04.377481 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f0afed-f095-4d9c-b77f-093466b68669 service placement] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:36:04.377817 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f0afed-f095-4d9c-b77f-093466b68669 service placement] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:36:04.378107 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f0afed-f095-4d9c-b77f-093466b68669 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:36:04.379823 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-f1f0afed-f095-4d9c-b77f-093466b68669 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:36:04.391466 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f0afed-f095-4d9c-b77f-093466b68669 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:36:04.393072 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f0afed-f095-4d9c-b77f-093466b68669 service placement] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:36:04.404839 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1447/2863] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 12:36:04 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 12:36:04.417932 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925aa6be-4500-4888-8a6c-d8c3e87d7fd4 service placement] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:36:04.481751 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925aa6be-4500-4888-8a6c-d8c3e87d7fd4 service placement] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:36:04.486257 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa652f1-a066-4e8c-8680-69f7b2f8f429 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:36:04.486920 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa652f1-a066-4e8c-8680-69f7b2f8f429 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:36:04.487202 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa652f1-a066-4e8c-8680-69f7b2f8f429 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:36:04.487474 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa652f1-a066-4e8c-8680-69f7b2f8f429 admin admin] PATH_INFO: `/v3/users/08bd9f54467341afacba19215b78bf59` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:36:04.488630 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa652f1-a066-4e8c-8680-69f7b2f8f429 admin admin] RBAC: Authorizing `identity:delete_user(user_id=08bd9f54467341afacba19215b78bf59)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:36:04.490556 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa652f1-a066-4e8c-8680-69f7b2f8f429 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:36:04.535942 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-faa652f1-a066-4e8c-8680-69f7b2f8f429 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '08bd9f54467341afacba19215b78bf59'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:36:04.540242 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:36:04.549095 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-faa652f1-a066-4e8c-8680-69f7b2f8f429 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '08bd9f54467341afacba19215b78bf59'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:36:04.555877 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-faa652f1-a066-4e8c-8680-69f7b2f8f429 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '08bd9f54467341afacba19215b78bf59'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:36:04.556481 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-faa652f1-a066-4e8c-8680-69f7b2f8f429 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '08bd9f54467341afacba19215b78bf59'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:36:04.563280 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-faa652f1-a066-4e8c-8680-69f7b2f8f429 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '08bd9f54467341afacba19215b78bf59'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:36:04.569524 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1417/2864] 10.222.0.15 () {58 vars in 1224 bytes} [Thu Sep 15 12:36:04 2022] DELETE /identity/v3/users/08bd9f54467341afacba19215b78bf59 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Sep 15 12:36:04.739554 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f0afed-f095-4d9c-b77f-093466b68669 service placement] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:36:04.751302 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f0afed-f095-4d9c-b77f-093466b68669 service placement] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:36:04.809369 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d88e14-2ec9-4218-90f8-13224653643c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:36:04.810001 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d88e14-2ec9-4218-90f8-13224653643c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:36:04.810279 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d88e14-2ec9-4218-90f8-13224653643c admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:36:04.810547 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d88e14-2ec9-4218-90f8-13224653643c admin admin] PATH_INFO: `/v3/projects/84b799b1cfab4245bf9fbfce0faf036a` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:36:04.811653 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d88e14-2ec9-4218-90f8-13224653643c admin admin] RBAC: Authorizing `identity:delete_project(project_id=84b799b1cfab4245bf9fbfce0faf036a)` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:36:04.813573 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d88e14-2ec9-4218-90f8-13224653643c admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:36:04.859533 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-61d88e14-2ec9-4218-90f8-13224653643c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '84b799b1cfab4245bf9fbfce0faf036a', 'request_id': 'req-61d88e14-2ec9-4218-90f8-13224653643c'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:36:04.867014 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-61d88e14-2ec9-4218-90f8-13224653643c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '84b799b1cfab4245bf9fbfce0faf036a', 'request_id': 'req-61d88e14-2ec9-4218-90f8-13224653643c'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:36:04.872500 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-61d88e14-2ec9-4218-90f8-13224653643c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '84b799b1cfab4245bf9fbfce0faf036a', 'request_id': 'req-61d88e14-2ec9-4218-90f8-13224653643c'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:36:04.884002 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-61d88e14-2ec9-4218-90f8-13224653643c admin admin] The token cache is being invalidate because project 84b799b1cfab4245bf9fbfce0faf036a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81449) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 12:36:04.884288 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-61d88e14-2ec9-4218-90f8-13224653643c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:36:04.884926 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-61d88e14-2ec9-4218-90f8-13224653643c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:36:04.886363 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1448/2865] 10.222.0.15 () {58 vars in 1233 bytes} [Thu Sep 15 12:36:04 2022] DELETE /identity/v3/projects/84b799b1cfab4245bf9fbfce0faf036a => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:36:04.890506 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa652f1-a066-4e8c-8680-69f7b2f8f429 admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:36:04.891994 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-faa652f1-a066-4e8c-8680-69f7b2f8f429 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:36:04.905104 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa652f1-a066-4e8c-8680-69f7b2f8f429 admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:36:04.961734 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46aa9c5d-1ee0-4e62-b4cb-1fe8c6f610ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:36:04.962297 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46aa9c5d-1ee0-4e62-b4cb-1fe8c6f610ad admin admin] REQUEST_METHOD: `DELETE` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:36:04.962578 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46aa9c5d-1ee0-4e62-b4cb-1fe8c6f610ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:36:04.962851 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46aa9c5d-1ee0-4e62-b4cb-1fe8c6f610ad admin admin] PATH_INFO: `/v3/users/81c5a3f4327b4301b47d9d0623280922` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:36:04.963973 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46aa9c5d-1ee0-4e62-b4cb-1fe8c6f610ad admin admin] RBAC: Authorizing `identity:delete_user(user_id=81c5a3f4327b4301b47d9d0623280922)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:36:04.965971 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46aa9c5d-1ee0-4e62-b4cb-1fe8c6f610ad admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:36:05.011548 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-46aa9c5d-1ee0-4e62-b4cb-1fe8c6f610ad admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '81c5a3f4327b4301b47d9d0623280922'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:36:05.023197 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-46aa9c5d-1ee0-4e62-b4cb-1fe8c6f610ad admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '81c5a3f4327b4301b47d9d0623280922'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:36:05.029791 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-46aa9c5d-1ee0-4e62-b4cb-1fe8c6f610ad admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '81c5a3f4327b4301b47d9d0623280922'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:36:05.030543 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-46aa9c5d-1ee0-4e62-b4cb-1fe8c6f610ad admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '81c5a3f4327b4301b47d9d0623280922'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:36:05.037614 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-46aa9c5d-1ee0-4e62-b4cb-1fe8c6f610ad admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '81c5a3f4327b4301b47d9d0623280922'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:36:05.043333 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1418/2866] 10.222.0.15 () {58 vars in 1224 bytes} [Thu Sep 15 12:36:04 2022] DELETE /identity/v3/users/81c5a3f4327b4301b47d9d0623280922 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:36:05.211901 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d88e14-2ec9-4218-90f8-13224653643c admin admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:36:05.213417 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-61d88e14-2ec9-4218-90f8-13224653643c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:36:05.227654 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d88e14-2ec9-4218-90f8-13224653643c admin admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:36:05.268863 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:36:05.287784 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d5676b-4571-484d-8f5b-286322288b14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:36:05.288418 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d5676b-4571-484d-8f5b-286322288b14 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:36:05.288704 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d5676b-4571-484d-8f5b-286322288b14 admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:36:05.289007 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d5676b-4571-484d-8f5b-286322288b14 admin admin] PATH_INFO: `/v3/projects/6e2586884a2a4eebb15dae6c061ac277` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:36:05.290132 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d5676b-4571-484d-8f5b-286322288b14 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6e2586884a2a4eebb15dae6c061ac277)` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:36:05.292017 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d5676b-4571-484d-8f5b-286322288b14 admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:36:05.337865 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-67d5676b-4571-484d-8f5b-286322288b14 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6e2586884a2a4eebb15dae6c061ac277', 'request_id': 'req-67d5676b-4571-484d-8f5b-286322288b14'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:36:05.345075 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-67d5676b-4571-484d-8f5b-286322288b14 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6e2586884a2a4eebb15dae6c061ac277', 'request_id': 'req-67d5676b-4571-484d-8f5b-286322288b14'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:36:05.351182 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-67d5676b-4571-484d-8f5b-286322288b14 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6e2586884a2a4eebb15dae6c061ac277', 'request_id': 'req-67d5676b-4571-484d-8f5b-286322288b14'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:36:05.361699 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-67d5676b-4571-484d-8f5b-286322288b14 admin admin] The token cache is being invalidate because project 6e2586884a2a4eebb15dae6c061ac277 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81449) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 12:36:05.362274 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-67d5676b-4571-484d-8f5b-286322288b14 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:36:05.362646 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-67d5676b-4571-484d-8f5b-286322288b14 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:36:05.366354 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1449/2867] 10.222.0.15 () {58 vars in 1233 bytes} [Thu Sep 15 12:36:05 2022] DELETE /identity/v3/projects/6e2586884a2a4eebb15dae6c061ac277 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:36:05.371385 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc399ae4-ac77-4b77-bc00-601c93f0fe1f None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:36:05.371671 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc399ae4-ac77-4b77-bc00-601c93f0fe1f None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:36:05.371943 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc399ae4-ac77-4b77-bc00-601c93f0fe1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:36:05.391483 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-fc399ae4-ac77-4b77-bc00-601c93f0fe1f None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:36:05.392681 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-fc399ae4-ac77-4b77-bc00-601c93f0fe1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:36:05.435976 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:DELETE=13,keystone:SELECT=85,keystone:INSERT=2 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:36:05.452513 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1419/2868] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 12:36:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:36:05.461576 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335c1694-b4e1-4e7f-af35-e2ece29a6f4b None None] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:36:05.461866 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335c1694-b4e1-4e7f-af35-e2ece29a6f4b None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:36:05.462188 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335c1694-b4e1-4e7f-af35-e2ece29a6f4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:36:05.481927 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.auth.core [None req-335c1694-b4e1-4e7f-af35-e2ece29a6f4b None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81449) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:36:05.483148 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-335c1694-b4e1-4e7f-af35-e2ece29a6f4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:36:05.491511 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1450/2869] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 12:36:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:36:05.495619 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc399ae4-ac77-4b77-bc00-601c93f0fe1f None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:36:05.505577 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc399ae4-ac77-4b77-bc00-601c93f0fe1f None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:36:05.510125 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a948d2e4-03b4-4d80-a0ec-ab7294d94f16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:36:05.510719 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a948d2e4-03b4-4d80-a0ec-ab7294d94f16 admin admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:36:05.510998 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a948d2e4-03b4-4d80-a0ec-ab7294d94f16 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:36:05.511267 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a948d2e4-03b4-4d80-a0ec-ab7294d94f16 admin admin] PATH_INFO: `/v3/domains` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:36:05.511900 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a948d2e4-03b4-4d80-a0ec-ab7294d94f16 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:36:05.512211 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a948d2e4-03b4-4d80-a0ec-ab7294d94f16 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:36:05.513805 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a948d2e4-03b4-4d80-a0ec-ab7294d94f16 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:36:05.525591 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1420/2870] 10.222.0.15 () {58 vars in 1153 bytes} [Thu Sep 15 12:36:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:36:05.529671 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335c1694-b4e1-4e7f-af35-e2ece29a6f4b None None] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:36:05.539046 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335c1694-b4e1-4e7f-af35-e2ece29a6f4b None None] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:36:05.543636 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca84f8e7-b698-4e0a-82c3-5da23e4af74b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:36:05.544222 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca84f8e7-b698-4e0a-82c3-5da23e4af74b admin admin] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:36:05.544499 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca84f8e7-b698-4e0a-82c3-5da23e4af74b admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:36:05.544788 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca84f8e7-b698-4e0a-82c3-5da23e4af74b admin admin] PATH_INFO: `/v3/projects` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:36:05.545446 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca84f8e7-b698-4e0a-82c3-5da23e4af74b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:36:05.547328 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca84f8e7-b698-4e0a-82c3-5da23e4af74b admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:36:05.583280 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1451/2871] 10.222.0.15 () {60 vars in 1153 bytes} [Thu Sep 15 12:36:05 2022] POST /identity/v3/projects => generated 371 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 15 12:36:05.587216 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a948d2e4-03b4-4d80-a0ec-ab7294d94f16 admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:36:05.596993 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a948d2e4-03b4-4d80-a0ec-ab7294d94f16 admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:36:05.651760 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb6903a-afce-4348-a19e-90534d8ab114 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:36:05.652449 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb6903a-afce-4348-a19e-90534d8ab114 admin admin] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:36:05.652707 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb6903a-afce-4348-a19e-90534d8ab114 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:36:05.653297 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb6903a-afce-4348-a19e-90534d8ab114 admin admin] PATH_INFO: `/v3/users` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:36:05.654025 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb6903a-afce-4348-a19e-90534d8ab114 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:36:05.655915 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb6903a-afce-4348-a19e-90534d8ab114 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:36:05.676854 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1421/2872] 10.222.0.15 () {60 vars in 1144 bytes} [Thu Sep 15 12:36:05 2022] POST /identity/v3/users => generated 286 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 12:36:05.680965 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca84f8e7-b698-4e0a-82c3-5da23e4af74b admin admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:36:05.692462 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca84f8e7-b698-4e0a-82c3-5da23e4af74b admin admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:36:05.697268 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a1d2ae-0e54-4032-b5ff-4bdf9713433d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:36:05.697884 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a1d2ae-0e54-4032-b5ff-4bdf9713433d admin admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:36:05.698190 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a1d2ae-0e54-4032-b5ff-4bdf9713433d admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:36:05.698488 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a1d2ae-0e54-4032-b5ff-4bdf9713433d admin admin] PATH_INFO: `/v3/roles` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:36:05.699130 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a1d2ae-0e54-4032-b5ff-4bdf9713433d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:36:05.700713 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a1d2ae-0e54-4032-b5ff-4bdf9713433d admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:36:05.715012 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1452/2873] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 12:36:05 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Sep 15 12:36:05.716792 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb6903a-afce-4348-a19e-90534d8ab114 admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:36:05.726941 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb6903a-afce-4348-a19e-90534d8ab114 admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:36:05.731639 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f0a054-56d5-4c5c-b4fc-db51c0b48972 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:36:05.732323 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f0a054-56d5-4c5c-b4fc-db51c0b48972 admin admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:36:05.732668 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f0a054-56d5-4c5c-b4fc-db51c0b48972 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:36:05.733033 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f0a054-56d5-4c5c-b4fc-db51c0b48972 admin admin] PATH_INFO: `/v3/roles` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:36:05.733806 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f0a054-56d5-4c5c-b4fc-db51c0b48972 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:36:05.735674 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f0a054-56d5-4c5c-b4fc-db51c0b48972 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:36:05.747200 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1422/2874] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 12:36:05 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 12:36:05.751516 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a1d2ae-0e54-4032-b5ff-4bdf9713433d admin admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:36:05.762640 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a1d2ae-0e54-4032-b5ff-4bdf9713433d admin admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:36:05.768981 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ff7495-4a36-49d9-8820-06837f565340 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:36:05.769684 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ff7495-4a36-49d9-8820-06837f565340 admin admin] REQUEST_METHOD: `PUT` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:36:05.769968 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ff7495-4a36-49d9-8820-06837f565340 admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:36:05.770238 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ff7495-4a36-49d9-8820-06837f565340 admin admin] PATH_INFO: `/v3/projects/d62b434017e447f88e0a1cf275ba27bb/users/d36ce80b9d3e4810be863d54a8bb7865/roles/0679d6628a25438d90dc4bdc0401eda2` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:36:05.784776 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ff7495-4a36-49d9-8820-06837f565340 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d62b434017e447f88e0a1cf275ba27bb, user_id=d36ce80b9d3e4810be863d54a8bb7865, role_id=0679d6628a25438d90dc4bdc0401eda2)` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:36:05.787451 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ff7495-4a36-49d9-8820-06837f565340 admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:36:05.799638 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1453/2875] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 15 12:36:05 2022] PUT /identity/v3/projects/d62b434017e447f88e0a1cf275ba27bb/users/d36ce80b9d3e4810be863d54a8bb7865/roles/0679d6628a25438d90dc4bdc0401eda2 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:36:05.805165 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fef6cd-c6b1-4c82-933e-9257822567f5 None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:36:05.805453 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fef6cd-c6b1-4c82-933e-9257822567f5 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:36:05.805725 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fef6cd-c6b1-4c82-933e-9257822567f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:36:05.824940 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-71fef6cd-c6b1-4c82-933e-9257822567f5 None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:36:05.826146 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-71fef6cd-c6b1-4c82-933e-9257822567f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:36:05.883746 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1423/2876] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 12:36:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:36:05.893309 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ff7495-4a36-49d9-8820-06837f565340 admin admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:36:05.895121 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-67ff7495-4a36-49d9-8820-06837f565340 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:36:05.950667 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ff7495-4a36-49d9-8820-06837f565340 admin admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:36:05.954238 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627d04de-f682-423d-8226-00ec5af08a64 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfbb4d6809064f9cb9e06626ed7d6814', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '540a32ef88764e8c8d9201ae22f6ddaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:36:05.954841 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627d04de-f682-423d-8226-00ec5af08a64 service neutron] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:36:05.955119 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627d04de-f682-423d-8226-00ec5af08a64 service neutron] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:36:05.955387 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627d04de-f682-423d-8226-00ec5af08a64 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:36:05.964814 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-627d04de-f682-423d-8226-00ec5af08a64 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:36:05.966301 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-627d04de-f682-423d-8226-00ec5af08a64 service neutron] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:36:05.976154 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1454/2877] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 12:36:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 12:36:09.847048 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bf6795-8886-490a-9e98-b41ce4f78ade None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:36:09.847607 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bf6795-8886-490a-9e98-b41ce4f78ade None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:36:09.847607 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bf6795-8886-490a-9e98-b41ce4f78ade None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:36:09.887039 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-e0bf6795-8886-490a-9e98-b41ce4f78ade None None] MFA Rules not processed for user `d36ce80b9d3e4810be863d54a8bb7865`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:36:09.888358 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-e0bf6795-8886-490a-9e98-b41ce4f78ade None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:36:09.938446 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:36:09.972843 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1424/2878] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 12:36:09 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:36:09.974365 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627d04de-f682-423d-8226-00ec5af08a64 service neutron] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:36:09.979788 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:36:09.985363 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627d04de-f682-423d-8226-00ec5af08a64 service neutron] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:36:09.993289 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c59bf7-71a7-476c-9c5b-251be11b9af7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:36:09.994066 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c59bf7-71a7-476c-9c5b-251be11b9af7 admin admin] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:36:09.994412 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c59bf7-71a7-476c-9c5b-251be11b9af7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:36:09.994785 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c59bf7-71a7-476c-9c5b-251be11b9af7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:36:09.995510 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c59bf7-71a7-476c-9c5b-251be11b9af7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:36:09.997541 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c59bf7-71a7-476c-9c5b-251be11b9af7 admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:36:10.032937 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0bf6795-8886-490a-9e98-b41ce4f78ade None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:36:10.038508 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1455/2879] 10.222.0.15 () {60 vars in 1153 bytes} [Thu Sep 15 12:36:09 2022] POST /identity/v3/projects => generated 373 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 12:36:10.044806 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0bf6795-8886-490a-9e98-b41ce4f78ade None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:36:10.107688 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dfea12d-a01c-4ac2-b3ae-be3427deb1d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:36:10.108307 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfea12d-a01c-4ac2-b3ae-be3427deb1d3 admin admin] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:36:10.108587 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfea12d-a01c-4ac2-b3ae-be3427deb1d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:36:10.108897 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfea12d-a01c-4ac2-b3ae-be3427deb1d3 admin admin] PATH_INFO: `/v3/users` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:36:10.109533 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dfea12d-a01c-4ac2-b3ae-be3427deb1d3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:36:10.111445 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dfea12d-a01c-4ac2-b3ae-be3427deb1d3 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:36:10.135051 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1425/2880] 10.222.0.15 () {60 vars in 1144 bytes} [Thu Sep 15 12:36:10 2022] POST /identity/v3/users => generated 293 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 12:36:10.139022 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c59bf7-71a7-476c-9c5b-251be11b9af7 admin admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:36:10.149743 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c59bf7-71a7-476c-9c5b-251be11b9af7 admin admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:36:10.158589 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748f7eb2-3cf7-4229-98bd-b8370cabce8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:36:10.159283 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748f7eb2-3cf7-4229-98bd-b8370cabce8b admin admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:36:10.159529 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748f7eb2-3cf7-4229-98bd-b8370cabce8b admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:36:10.159869 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748f7eb2-3cf7-4229-98bd-b8370cabce8b admin admin] PATH_INFO: `/v3/roles` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:36:10.160425 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748f7eb2-3cf7-4229-98bd-b8370cabce8b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:36:10.162163 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748f7eb2-3cf7-4229-98bd-b8370cabce8b admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:36:10.174261 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1456/2881] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 12:36:10 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 12:36:10.178709 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dfea12d-a01c-4ac2-b3ae-be3427deb1d3 admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:36:10.188573 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dfea12d-a01c-4ac2-b3ae-be3427deb1d3 admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:36:10.193381 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd138211-a897-4464-a02c-ffa6731d9d5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:36:10.194042 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd138211-a897-4464-a02c-ffa6731d9d5c admin admin] REQUEST_METHOD: `PUT` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:36:10.194326 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd138211-a897-4464-a02c-ffa6731d9d5c admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:36:10.194604 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd138211-a897-4464-a02c-ffa6731d9d5c admin admin] PATH_INFO: `/v3/projects/21f7c584108b42eba04f4fb8106a5bfc/users/a977adee73094126b976d12fa8d09759/roles/9c769b5c82884766b2c510b1f0f1364c` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:36:10.211125 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd138211-a897-4464-a02c-ffa6731d9d5c admin admin] RBAC: Authorizing `identity:create_grant(project_id=21f7c584108b42eba04f4fb8106a5bfc, user_id=a977adee73094126b976d12fa8d09759, role_id=9c769b5c82884766b2c510b1f0f1364c)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:36:10.213944 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd138211-a897-4464-a02c-ffa6731d9d5c admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:36:10.228273 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1426/2882] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 15 12:36:10 2022] PUT /identity/v3/projects/21f7c584108b42eba04f4fb8106a5bfc/users/a977adee73094126b976d12fa8d09759/roles/9c769b5c82884766b2c510b1f0f1364c => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:36:10.232135 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748f7eb2-3cf7-4229-98bd-b8370cabce8b admin admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:36:10.242434 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748f7eb2-3cf7-4229-98bd-b8370cabce8b admin admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:36:10.299408 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643ae5a6-d17c-4da9-96ae-40beb97b5a19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:36:10.300015 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643ae5a6-d17c-4da9-96ae-40beb97b5a19 admin admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:36:10.300317 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643ae5a6-d17c-4da9-96ae-40beb97b5a19 admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:36:10.300588 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643ae5a6-d17c-4da9-96ae-40beb97b5a19 admin admin] PATH_INFO: `/v3/roles` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:36:10.301283 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643ae5a6-d17c-4da9-96ae-40beb97b5a19 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:36:10.302891 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643ae5a6-d17c-4da9-96ae-40beb97b5a19 admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:36:10.312468 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1457/2883] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 12:36:10 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 12:36:10.316660 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd138211-a897-4464-a02c-ffa6731d9d5c admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:36:10.326910 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd138211-a897-4464-a02c-ffa6731d9d5c admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:36:10.331570 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc06058-3897-414c-97b9-8fac4580a225 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:36:10.333030 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc06058-3897-414c-97b9-8fac4580a225 admin admin] REQUEST_METHOD: `PUT` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:36:10.333311 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc06058-3897-414c-97b9-8fac4580a225 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:36:10.333580 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc06058-3897-414c-97b9-8fac4580a225 admin admin] PATH_INFO: `/v3/projects/21f7c584108b42eba04f4fb8106a5bfc/users/a977adee73094126b976d12fa8d09759/roles/9c769b5c82884766b2c510b1f0f1364c` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:36:10.335511 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc06058-3897-414c-97b9-8fac4580a225 admin admin] RBAC: Authorizing `identity:create_grant(project_id=21f7c584108b42eba04f4fb8106a5bfc, user_id=a977adee73094126b976d12fa8d09759, role_id=9c769b5c82884766b2c510b1f0f1364c)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:36:10.338196 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc06058-3897-414c-97b9-8fac4580a225 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:36:10.349527 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1427/2884] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 15 12:36:10 2022] PUT /identity/v3/projects/21f7c584108b42eba04f4fb8106a5bfc/users/a977adee73094126b976d12fa8d09759/roles/9c769b5c82884766b2c510b1f0f1364c => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:36:14.750686 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e9924f-fe1c-49ec-824e-a4612f00bc8f None None] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:36:14.750686 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e9924f-fe1c-49ec-824e-a4612f00bc8f None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:36:14.750686 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e9924f-fe1c-49ec-824e-a4612f00bc8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:36:14.785238 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.auth.core [None req-62e9924f-fe1c-49ec-824e-a4612f00bc8f None None] MFA Rules not processed for user `a977adee73094126b976d12fa8d09759`. Rule list: `[]` (Enabled: `True`). {{(pid=81449) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:36:14.786497 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-62e9924f-fe1c-49ec-824e-a4612f00bc8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:36:14.859185 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1458/2885] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 12:36:14 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:36:14.876044 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc06058-3897-414c-97b9-8fac4580a225 admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:36:14.881113 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-9bc06058-3897-414c-97b9-8fac4580a225 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:36:14.964507 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc06058-3897-414c-97b9-8fac4580a225 admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:36:14.970057 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb97b830-a3d5-41d1-a5d3-47204a49a6ec service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eface89e261840a2abb4206be26455c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '540a32ef88764e8c8d9201ae22f6ddaf', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:36:14.974249 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb97b830-a3d5-41d1-a5d3-47204a49a6ec service nova] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:36:14.974249 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb97b830-a3d5-41d1-a5d3-47204a49a6ec service nova] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:36:14.974249 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb97b830-a3d5-41d1-a5d3-47204a49a6ec service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:36:15.007542 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:36:15.034536 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb97b830-a3d5-41d1-a5d3-47204a49a6ec service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:36:15.036670 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb97b830-a3d5-41d1-a5d3-47204a49a6ec service nova] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:36:15.046542 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1428/2886] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 12:36:14 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 12:36:16.035732 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8593c4f2-4a0a-4ce0-b5ae-82492fd934aa None None] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:36:16.036268 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8593c4f2-4a0a-4ce0-b5ae-82492fd934aa None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:36:16.036268 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8593c4f2-4a0a-4ce0-b5ae-82492fd934aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:36:16.055580 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.auth.core [None req-8593c4f2-4a0a-4ce0-b5ae-82492fd934aa None None] MFA Rules not processed for user `d36ce80b9d3e4810be863d54a8bb7865`. Rule list: `[]` (Enabled: `True`). {{(pid=81449) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:36:16.056819 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-8593c4f2-4a0a-4ce0-b5ae-82492fd934aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:36:16.068414 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1459/2887] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 12:36:16 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:36:25.042073 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=12 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:36:26.049578 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=79,keystone:INSERT=1 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:36:32.638345 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb97b830-a3d5-41d1-a5d3-47204a49a6ec service nova] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:36:32.692031 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb97b830-a3d5-41d1-a5d3-47204a49a6ec service nova] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:36:32.695692 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f33b7a-a791-45c0-bbe1-76c677e20a9c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfbb4d6809064f9cb9e06626ed7d6814', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '540a32ef88764e8c8d9201ae22f6ddaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:36:32.696297 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f33b7a-a791-45c0-bbe1-76c677e20a9c service neutron] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:36:32.696578 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f33b7a-a791-45c0-bbe1-76c677e20a9c service neutron] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:36:32.696893 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f33b7a-a791-45c0-bbe1-76c677e20a9c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:36:32.698592 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-71f33b7a-a791-45c0-bbe1-76c677e20a9c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:36:32.710109 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f33b7a-a791-45c0-bbe1-76c677e20a9c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:36:32.711523 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f33b7a-a791-45c0-bbe1-76c677e20a9c service neutron] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:36:32.719647 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1429/2888] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 12:36:32 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 12:36:33.168477 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8593c4f2-4a0a-4ce0-b5ae-82492fd934aa None None] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:36:33.169900 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-8593c4f2-4a0a-4ce0-b5ae-82492fd934aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:36:33.238442 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8593c4f2-4a0a-4ce0-b5ae-82492fd934aa None None] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:36:33.243321 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843552e2-51e3-4e30-b43c-806c84c693ac service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f64a058bd5440cb5268f228aa6e392', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '540a32ef88764e8c8d9201ae22f6ddaf', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:36:33.243923 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843552e2-51e3-4e30-b43c-806c84c693ac service placement] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:36:33.244199 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843552e2-51e3-4e30-b43c-806c84c693ac service placement] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:36:33.244467 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843552e2-51e3-4e30-b43c-806c84c693ac service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:36:33.246187 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-843552e2-51e3-4e30-b43c-806c84c693ac service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:36:33.257913 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-843552e2-51e3-4e30-b43c-806c84c693ac service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:36:33.259501 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-843552e2-51e3-4e30-b43c-806c84c693ac service placement] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:36:33.269784 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1460/2889] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 12:36:33 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 12:36:42.716015 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=26 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:36:43.265294 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=32 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:36:44.608084 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f33b7a-a791-45c0-bbe1-76c677e20a9c service neutron] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:36:44.609639 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-71f33b7a-a791-45c0-bbe1-76c677e20a9c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:36:44.626274 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f33b7a-a791-45c0-bbe1-76c677e20a9c service neutron] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:36:44.632240 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5f305d-2dd4-4d88-b93e-310961fef622 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eface89e261840a2abb4206be26455c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '540a32ef88764e8c8d9201ae22f6ddaf', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:36:44.632872 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5f305d-2dd4-4d88-b93e-310961fef622 service nova] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:36:44.633153 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5f305d-2dd4-4d88-b93e-310961fef622 service nova] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:36:44.633420 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5f305d-2dd4-4d88-b93e-310961fef622 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:36:44.635113 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-be5f305d-2dd4-4d88-b93e-310961fef622 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:36:44.647083 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5f305d-2dd4-4d88-b93e-310961fef622 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:36:44.648673 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5f305d-2dd4-4d88-b93e-310961fef622 service nova] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:36:44.659794 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1430/2890] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 12:36:44 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 12:36:47.414955 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843552e2-51e3-4e30-b43c-806c84c693ac service placement] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:36:47.424257 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843552e2-51e3-4e30-b43c-806c84c693ac service placement] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:36:47.428341 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d7e936-482c-4f52-b96f-0b471c9ee818 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfbb4d6809064f9cb9e06626ed7d6814', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '540a32ef88764e8c8d9201ae22f6ddaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:36:47.428963 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d7e936-482c-4f52-b96f-0b471c9ee818 service neutron] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:36:47.429240 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d7e936-482c-4f52-b96f-0b471c9ee818 service neutron] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:36:47.429504 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d7e936-482c-4f52-b96f-0b471c9ee818 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:36:47.438035 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d7e936-482c-4f52-b96f-0b471c9ee818 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:36:47.439434 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d7e936-482c-4f52-b96f-0b471c9ee818 service neutron] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:36:47.448548 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1461/2891] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 12:36:47 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 12:36:54.654653 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=6 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:36:57.444519 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=6 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:37:25.440215 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5f305d-2dd4-4d88-b93e-310961fef622 service nova] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:37:25.451517 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5f305d-2dd4-4d88-b93e-310961fef622 service nova] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:37:25.457433 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1da677-3702-46ed-80b1-f1aa3b9c350d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f64a058bd5440cb5268f228aa6e392', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '540a32ef88764e8c8d9201ae22f6ddaf', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:37:25.458029 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1da677-3702-46ed-80b1-f1aa3b9c350d service placement] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:37:25.458552 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1da677-3702-46ed-80b1-f1aa3b9c350d service placement] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:37:25.458829 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1da677-3702-46ed-80b1-f1aa3b9c350d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:37:25.460603 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-fc1da677-3702-46ed-80b1-f1aa3b9c350d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:37:25.471766 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1da677-3702-46ed-80b1-f1aa3b9c350d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:37:25.473833 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1da677-3702-46ed-80b1-f1aa3b9c350d service placement] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:37:25.483672 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1431/2892] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 12:37:25 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 12:37:35.479243 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=6 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:39:01.096874 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d7e936-482c-4f52-b96f-0b471c9ee818 service neutron] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:39:01.106049 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d7e936-482c-4f52-b96f-0b471c9ee818 service neutron] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:39:01.109788 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50317ba4-58f1-44fa-bccf-d11abb4d9ecc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfbb4d6809064f9cb9e06626ed7d6814', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '540a32ef88764e8c8d9201ae22f6ddaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:39:01.110446 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50317ba4-58f1-44fa-bccf-d11abb4d9ecc service neutron] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:39:01.110755 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50317ba4-58f1-44fa-bccf-d11abb4d9ecc service neutron] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:39:01.111025 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50317ba4-58f1-44fa-bccf-d11abb4d9ecc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:39:01.112772 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-50317ba4-58f1-44fa-bccf-d11abb4d9ecc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:39:01.123137 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50317ba4-58f1-44fa-bccf-d11abb4d9ecc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:39:01.124561 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50317ba4-58f1-44fa-bccf-d11abb4d9ecc service neutron] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:39:01.133347 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1462/2893] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 12:39:01 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Sep 15 12:39:01.253501 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1da677-3702-46ed-80b1-f1aa3b9c350d service placement] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:39:01.263366 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1da677-3702-46ed-80b1-f1aa3b9c350d service placement] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:39:01.267322 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3bce6d-6435-4f4a-a7b9-c8eb6c5487a3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfbb4d6809064f9cb9e06626ed7d6814', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '540a32ef88764e8c8d9201ae22f6ddaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:39:01.267933 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3bce6d-6435-4f4a-a7b9-c8eb6c5487a3 service neutron] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:39:01.268495 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3bce6d-6435-4f4a-a7b9-c8eb6c5487a3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:39:01.268495 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3bce6d-6435-4f4a-a7b9-c8eb6c5487a3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:39:01.283340 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-ff3bce6d-6435-4f4a-a7b9-c8eb6c5487a3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:39:01.296481 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3bce6d-6435-4f4a-a7b9-c8eb6c5487a3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:39:01.297936 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3bce6d-6435-4f4a-a7b9-c8eb6c5487a3 service neutron] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:39:01.309140 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1432/2894] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 12:39:01 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 12:39:11.129077 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=6 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:39:11.304048 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=6 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:39:11.736399 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50317ba4-58f1-44fa-bccf-d11abb4d9ecc service neutron] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:39:11.745282 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50317ba4-58f1-44fa-bccf-d11abb4d9ecc service neutron] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:39:11.749015 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8c22f1-43d0-4d85-aec6-a135d70ff99e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfbb4d6809064f9cb9e06626ed7d6814', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '540a32ef88764e8c8d9201ae22f6ddaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:39:11.749615 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8c22f1-43d0-4d85-aec6-a135d70ff99e service neutron] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:39:11.749897 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8c22f1-43d0-4d85-aec6-a135d70ff99e service neutron] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:39:11.750170 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8c22f1-43d0-4d85-aec6-a135d70ff99e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:39:11.751909 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-2a8c22f1-43d0-4d85-aec6-a135d70ff99e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:39:11.767128 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8c22f1-43d0-4d85-aec6-a135d70ff99e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:39:11.768700 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8c22f1-43d0-4d85-aec6-a135d70ff99e service neutron] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:39:11.780997 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1463/2895] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 12:39:11 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 12:39:21.775658 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=6 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:40:28.541836 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3bce6d-6435-4f4a-a7b9-c8eb6c5487a3 service neutron] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:40:28.544061 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-ff3bce6d-6435-4f4a-a7b9-c8eb6c5487a3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:40:28.557480 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3bce6d-6435-4f4a-a7b9-c8eb6c5487a3 service neutron] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:40:28.561259 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b175ac8-37af-4d45-bd12-78e432b06b06 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfbb4d6809064f9cb9e06626ed7d6814', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '540a32ef88764e8c8d9201ae22f6ddaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:40:28.562282 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b175ac8-37af-4d45-bd12-78e432b06b06 service neutron] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:40:28.562739 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b175ac8-37af-4d45-bd12-78e432b06b06 service neutron] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:40:28.562922 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b175ac8-37af-4d45-bd12-78e432b06b06 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:40:28.564853 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-2b175ac8-37af-4d45-bd12-78e432b06b06 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:40:28.579249 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b175ac8-37af-4d45-bd12-78e432b06b06 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:40:28.580693 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b175ac8-37af-4d45-bd12-78e432b06b06 service neutron] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:40:28.601168 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1433/2896] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 12:40:28 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 12:40:28.755733 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8c22f1-43d0-4d85-aec6-a135d70ff99e service neutron] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:40:28.765330 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8c22f1-43d0-4d85-aec6-a135d70ff99e service neutron] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:40:28.769587 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b53de2-6755-4c55-8910-65e8edbda4e1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfbb4d6809064f9cb9e06626ed7d6814', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '540a32ef88764e8c8d9201ae22f6ddaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:40:28.770194 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b53de2-6755-4c55-8910-65e8edbda4e1 service neutron] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:40:28.770473 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b53de2-6755-4c55-8910-65e8edbda4e1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:40:28.770741 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b53de2-6755-4c55-8910-65e8edbda4e1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:40:28.772586 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-00b53de2-6755-4c55-8910-65e8edbda4e1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:40:28.787683 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b53de2-6755-4c55-8910-65e8edbda4e1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:40:28.790655 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b53de2-6755-4c55-8910-65e8edbda4e1 service neutron] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:40:28.803754 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1464/2897] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 12:40:28 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 12:40:38.585925 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=6 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:40:38.798434 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=6 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:40:39.608399 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b175ac8-37af-4d45-bd12-78e432b06b06 service neutron] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:40:39.671263 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b175ac8-37af-4d45-bd12-78e432b06b06 service neutron] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:40:39.676409 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb390ed-02bd-46c3-acc5-212dafafca3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:40:39.677076 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb390ed-02bd-46c3-acc5-212dafafca3d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:40:39.677359 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb390ed-02bd-46c3-acc5-212dafafca3d admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:40:39.677629 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb390ed-02bd-46c3-acc5-212dafafca3d admin admin] PATH_INFO: `/v3/users/d36ce80b9d3e4810be863d54a8bb7865` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:40:39.678763 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb390ed-02bd-46c3-acc5-212dafafca3d admin admin] RBAC: Authorizing `identity:delete_user(user_id=d36ce80b9d3e4810be863d54a8bb7865)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:40:39.680672 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb390ed-02bd-46c3-acc5-212dafafca3d admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:40:39.725391 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-abb390ed-02bd-46c3-acc5-212dafafca3d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd36ce80b9d3e4810be863d54a8bb7865'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:40:39.736997 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-abb390ed-02bd-46c3-acc5-212dafafca3d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd36ce80b9d3e4810be863d54a8bb7865'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:40:39.745432 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-abb390ed-02bd-46c3-acc5-212dafafca3d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd36ce80b9d3e4810be863d54a8bb7865'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:40:39.746035 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-abb390ed-02bd-46c3-acc5-212dafafca3d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd36ce80b9d3e4810be863d54a8bb7865'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:40:39.754276 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-abb390ed-02bd-46c3-acc5-212dafafca3d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd36ce80b9d3e4810be863d54a8bb7865'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:40:39.760317 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1434/2898] 10.222.0.15 () {58 vars in 1224 bytes} [Thu Sep 15 12:40:39 2022] DELETE /identity/v3/users/d36ce80b9d3e4810be863d54a8bb7865 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:40:39.938150 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b53de2-6755-4c55-8910-65e8edbda4e1 service neutron] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:40:39.962143 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b53de2-6755-4c55-8910-65e8edbda4e1 service neutron] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:40:40.019802 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba75000c-6749-4eb9-bf7f-a8b0b359f5e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:40:40.020433 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba75000c-6749-4eb9-bf7f-a8b0b359f5e7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:40:40.020714 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba75000c-6749-4eb9-bf7f-a8b0b359f5e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:40:40.021014 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba75000c-6749-4eb9-bf7f-a8b0b359f5e7 admin admin] PATH_INFO: `/v3/projects/d62b434017e447f88e0a1cf275ba27bb` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:40:40.022129 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba75000c-6749-4eb9-bf7f-a8b0b359f5e7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d62b434017e447f88e0a1cf275ba27bb)` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:40:40.024085 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba75000c-6749-4eb9-bf7f-a8b0b359f5e7 admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:40:40.069659 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-ba75000c-6749-4eb9-bf7f-a8b0b359f5e7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd62b434017e447f88e0a1cf275ba27bb', 'request_id': 'req-ba75000c-6749-4eb9-bf7f-a8b0b359f5e7'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:40:40.077637 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-ba75000c-6749-4eb9-bf7f-a8b0b359f5e7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd62b434017e447f88e0a1cf275ba27bb', 'request_id': 'req-ba75000c-6749-4eb9-bf7f-a8b0b359f5e7'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:40:40.083755 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-ba75000c-6749-4eb9-bf7f-a8b0b359f5e7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd62b434017e447f88e0a1cf275ba27bb', 'request_id': 'req-ba75000c-6749-4eb9-bf7f-a8b0b359f5e7'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:40:40.095299 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-ba75000c-6749-4eb9-bf7f-a8b0b359f5e7 admin admin] The token cache is being invalidate because project d62b434017e447f88e0a1cf275ba27bb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81449) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 12:40:40.095584 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-ba75000c-6749-4eb9-bf7f-a8b0b359f5e7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:40:40.096209 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-ba75000c-6749-4eb9-bf7f-a8b0b359f5e7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:40:40.098384 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1465/2899] 10.222.0.15 () {58 vars in 1233 bytes} [Thu Sep 15 12:40:39 2022] DELETE /identity/v3/projects/d62b434017e447f88e0a1cf275ba27bb => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:40:40.103794 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb390ed-02bd-46c3-acc5-212dafafca3d admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:40:40.105349 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-abb390ed-02bd-46c3-acc5-212dafafca3d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:40:40.119946 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb390ed-02bd-46c3-acc5-212dafafca3d admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:40:40.175774 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6b4edf-71d8-40f6-96ef-aea3b5379f64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:40:40.176414 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6b4edf-71d8-40f6-96ef-aea3b5379f64 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:40:40.176691 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6b4edf-71d8-40f6-96ef-aea3b5379f64 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:40:40.176991 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6b4edf-71d8-40f6-96ef-aea3b5379f64 admin admin] PATH_INFO: `/v3/users/a977adee73094126b976d12fa8d09759` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:40:40.178116 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6b4edf-71d8-40f6-96ef-aea3b5379f64 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a977adee73094126b976d12fa8d09759)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:40:40.180016 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6b4edf-71d8-40f6-96ef-aea3b5379f64 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:40:40.200654 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:40:40.227393 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-3a6b4edf-71d8-40f6-96ef-aea3b5379f64 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a977adee73094126b976d12fa8d09759'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:40:40.239236 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-3a6b4edf-71d8-40f6-96ef-aea3b5379f64 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a977adee73094126b976d12fa8d09759'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:40:40.246583 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-3a6b4edf-71d8-40f6-96ef-aea3b5379f64 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a977adee73094126b976d12fa8d09759'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:40:40.247205 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-3a6b4edf-71d8-40f6-96ef-aea3b5379f64 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a977adee73094126b976d12fa8d09759'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:40:40.254076 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-3a6b4edf-71d8-40f6-96ef-aea3b5379f64 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a977adee73094126b976d12fa8d09759'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:40:40.259958 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1435/2900] 10.222.0.15 () {58 vars in 1224 bytes} [Thu Sep 15 12:40:40 2022] DELETE /identity/v3/users/a977adee73094126b976d12fa8d09759 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:40:40.432355 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba75000c-6749-4eb9-bf7f-a8b0b359f5e7 admin admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:40:40.434108 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-ba75000c-6749-4eb9-bf7f-a8b0b359f5e7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:40:40.447158 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba75000c-6749-4eb9-bf7f-a8b0b359f5e7 admin admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:40:40.504290 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f512db7f-6c91-40d4-83e8-c8e056fcbb58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:40:40.504955 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f512db7f-6c91-40d4-83e8-c8e056fcbb58 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:40:40.505236 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f512db7f-6c91-40d4-83e8-c8e056fcbb58 admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:40:40.505508 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f512db7f-6c91-40d4-83e8-c8e056fcbb58 admin admin] PATH_INFO: `/v3/projects/21f7c584108b42eba04f4fb8106a5bfc` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:40:40.506627 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f512db7f-6c91-40d4-83e8-c8e056fcbb58 admin admin] RBAC: Authorizing `identity:delete_project(project_id=21f7c584108b42eba04f4fb8106a5bfc)` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:40:40.508542 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f512db7f-6c91-40d4-83e8-c8e056fcbb58 admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:40:40.542721 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:40:40.556365 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-f512db7f-6c91-40d4-83e8-c8e056fcbb58 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '21f7c584108b42eba04f4fb8106a5bfc', 'request_id': 'req-f512db7f-6c91-40d4-83e8-c8e056fcbb58'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:40:40.563550 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-f512db7f-6c91-40d4-83e8-c8e056fcbb58 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '21f7c584108b42eba04f4fb8106a5bfc', 'request_id': 'req-f512db7f-6c91-40d4-83e8-c8e056fcbb58'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:40:40.569243 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-f512db7f-6c91-40d4-83e8-c8e056fcbb58 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '21f7c584108b42eba04f4fb8106a5bfc', 'request_id': 'req-f512db7f-6c91-40d4-83e8-c8e056fcbb58'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:40:40.579744 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-f512db7f-6c91-40d4-83e8-c8e056fcbb58 admin admin] The token cache is being invalidate because project 21f7c584108b42eba04f4fb8106a5bfc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81449) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 12:40:40.580026 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-f512db7f-6c91-40d4-83e8-c8e056fcbb58 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:40:40.580635 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-f512db7f-6c91-40d4-83e8-c8e056fcbb58 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:40:40.582112 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1466/2901] 10.222.0.15 () {58 vars in 1233 bytes} [Thu Sep 15 12:40:40 2022] DELETE /identity/v3/projects/21f7c584108b42eba04f4fb8106a5bfc => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:40:40.587297 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b3e823-ccd9-418b-9303-f6ff71858f6e None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:40:40.587581 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b3e823-ccd9-418b-9303-f6ff71858f6e None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:40:40.587853 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b3e823-ccd9-418b-9303-f6ff71858f6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:40:40.607896 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-88b3e823-ccd9-418b-9303-f6ff71858f6e None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:40:40.609127 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-88b3e823-ccd9-418b-9303-f6ff71858f6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:40:40.666247 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1436/2902] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 12:40:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:40:40.674982 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2832607-6aba-4653-a05b-b265ceab89e0 None None] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:40:40.675309 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2832607-6aba-4653-a05b-b265ceab89e0 None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:40:40.675579 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2832607-6aba-4653-a05b-b265ceab89e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:40:40.695198 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.auth.core [None req-d2832607-6aba-4653-a05b-b265ceab89e0 None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81449) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:40:40.696405 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-d2832607-6aba-4653-a05b-b265ceab89e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:40:40.704023 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1467/2903] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 12:40:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:40:40.708839 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b3e823-ccd9-418b-9303-f6ff71858f6e None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:40:40.718893 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b3e823-ccd9-418b-9303-f6ff71858f6e None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:40:40.723633 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c587b205-0b80-47da-aa75-d89996842a57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:40:40.724400 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c587b205-0b80-47da-aa75-d89996842a57 admin admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:40:40.724719 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c587b205-0b80-47da-aa75-d89996842a57 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:40:40.725089 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c587b205-0b80-47da-aa75-d89996842a57 admin admin] PATH_INFO: `/v3/domains` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:40:40.725818 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c587b205-0b80-47da-aa75-d89996842a57 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81448) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Sep 15 12:40:40.726188 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c587b205-0b80-47da-aa75-d89996842a57 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:40:40.727910 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c587b205-0b80-47da-aa75-d89996842a57 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:40:40.739944 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1437/2904] 10.222.0.15 () {58 vars in 1153 bytes} [Thu Sep 15 12:40:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Sep 15 12:40:40.744079 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2832607-6aba-4653-a05b-b265ceab89e0 None None] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:40:40.754067 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2832607-6aba-4653-a05b-b265ceab89e0 None None] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:40:40.759158 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206b2ed3-aacd-45d1-a133-b33fb34c5d9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:40:40.759742 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206b2ed3-aacd-45d1-a133-b33fb34c5d9f admin admin] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:40:40.760015 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206b2ed3-aacd-45d1-a133-b33fb34c5d9f admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:40:40.760278 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206b2ed3-aacd-45d1-a133-b33fb34c5d9f admin admin] PATH_INFO: `/v3/projects` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:40:40.760944 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-206b2ed3-aacd-45d1-a133-b33fb34c5d9f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:40:40.762800 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-206b2ed3-aacd-45d1-a133-b33fb34c5d9f admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:40:40.789203 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1468/2905] 10.222.0.15 () {60 vars in 1153 bytes} [Thu Sep 15 12:40:40 2022] POST /identity/v3/projects => generated 377 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Sep 15 12:40:40.793336 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c587b205-0b80-47da-aa75-d89996842a57 admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:40:40.804039 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c587b205-0b80-47da-aa75-d89996842a57 admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:40:40.860854 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad1fcc7-4c24-49ac-8467-ffb425836587 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:40:40.861466 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad1fcc7-4c24-49ac-8467-ffb425836587 admin admin] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:40:40.861753 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad1fcc7-4c24-49ac-8467-ffb425836587 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:40:40.862056 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad1fcc7-4c24-49ac-8467-ffb425836587 admin admin] PATH_INFO: `/v3/users` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:40:40.863225 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad1fcc7-4c24-49ac-8467-ffb425836587 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:40:40.865165 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad1fcc7-4c24-49ac-8467-ffb425836587 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:40:40.886532 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1438/2906] 10.222.0.15 () {60 vars in 1144 bytes} [Thu Sep 15 12:40:40 2022] POST /identity/v3/users => generated 289 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 12:40:40.890727 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206b2ed3-aacd-45d1-a133-b33fb34c5d9f admin admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:40:40.900914 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206b2ed3-aacd-45d1-a133-b33fb34c5d9f admin admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:40:40.905427 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223780c5-2f54-478e-9325-5e2ddc2466c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:40:40.906053 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223780c5-2f54-478e-9325-5e2ddc2466c9 admin admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:40:40.906327 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223780c5-2f54-478e-9325-5e2ddc2466c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:40:40.906593 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223780c5-2f54-478e-9325-5e2ddc2466c9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:40:40.907251 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-223780c5-2f54-478e-9325-5e2ddc2466c9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:40:40.908857 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-223780c5-2f54-478e-9325-5e2ddc2466c9 admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:40:40.918649 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1469/2907] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 12:40:40 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 12:40:40.922390 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad1fcc7-4c24-49ac-8467-ffb425836587 admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:40:40.932584 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad1fcc7-4c24-49ac-8467-ffb425836587 admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:40:40.937127 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de2f9e2-e303-4876-81ed-f4796c11c781 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:40:40.937714 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de2f9e2-e303-4876-81ed-f4796c11c781 admin admin] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:40:40.937997 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de2f9e2-e303-4876-81ed-f4796c11c781 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:40:40.938269 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de2f9e2-e303-4876-81ed-f4796c11c781 admin admin] PATH_INFO: `/v3/roles` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:40:40.938897 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de2f9e2-e303-4876-81ed-f4796c11c781 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:40:40.941088 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de2f9e2-e303-4876-81ed-f4796c11c781 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:40:40.951424 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1439/2908] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 12:40:40 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 12:40:40.955348 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223780c5-2f54-478e-9325-5e2ddc2466c9 admin admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:40:40.965771 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223780c5-2f54-478e-9325-5e2ddc2466c9 admin admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:40:40.970402 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83e251c-e3d7-4c46-8f1e-a0b78103c131 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:40:40.971179 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83e251c-e3d7-4c46-8f1e-a0b78103c131 admin admin] REQUEST_METHOD: `PUT` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:40:40.971460 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83e251c-e3d7-4c46-8f1e-a0b78103c131 admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:40:40.971731 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83e251c-e3d7-4c46-8f1e-a0b78103c131 admin admin] PATH_INFO: `/v3/projects/923986fe38f542549682ddff9eb63ff1/users/593948925ca642c196cbdef04985374f/roles/0679d6628a25438d90dc4bdc0401eda2` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:40:40.987028 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a83e251c-e3d7-4c46-8f1e-a0b78103c131 admin admin] RBAC: Authorizing `identity:create_grant(project_id=923986fe38f542549682ddff9eb63ff1, user_id=593948925ca642c196cbdef04985374f, role_id=0679d6628a25438d90dc4bdc0401eda2)` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:40:40.990199 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a83e251c-e3d7-4c46-8f1e-a0b78103c131 admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:40:41.002634 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1470/2909] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 15 12:40:40 2022] PUT /identity/v3/projects/923986fe38f542549682ddff9eb63ff1/users/593948925ca642c196cbdef04985374f/roles/0679d6628a25438d90dc4bdc0401eda2 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:40:41.008188 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f16659d-3424-48c0-9fbf-c0b483f671f3 None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:40:41.008474 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f16659d-3424-48c0-9fbf-c0b483f671f3 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:40:41.008749 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f16659d-3424-48c0-9fbf-c0b483f671f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:40:41.016684 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:DELETE=7,keystone:SELECT=89,keystone:INSERT=4 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:40:41.029304 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-8f16659d-3424-48c0-9fbf-c0b483f671f3 None None] MFA Rules not processed for user `c66eff9fe7124011868989f16846a501`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:40:41.031049 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-8f16659d-3424-48c0-9fbf-c0b483f671f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:40:41.088190 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1440/2910] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 12:40:41 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:40:41.098369 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83e251c-e3d7-4c46-8f1e-a0b78103c131 admin admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:40:41.100049 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-a83e251c-e3d7-4c46-8f1e-a0b78103c131 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:40:41.155994 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83e251c-e3d7-4c46-8f1e-a0b78103c131 admin admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:40:41.160176 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68b34fa-c94b-4804-9c20-9dc8cd25c561 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfbb4d6809064f9cb9e06626ed7d6814', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '540a32ef88764e8c8d9201ae22f6ddaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:40:41.160798 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68b34fa-c94b-4804-9c20-9dc8cd25c561 service neutron] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:40:41.161098 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68b34fa-c94b-4804-9c20-9dc8cd25c561 service neutron] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:40:41.161344 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68b34fa-c94b-4804-9c20-9dc8cd25c561 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:40:41.170290 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68b34fa-c94b-4804-9c20-9dc8cd25c561 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:40:41.171677 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68b34fa-c94b-4804-9c20-9dc8cd25c561 service neutron] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:40:41.181457 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1471/2911] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 12:40:41 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 12:40:45.276669 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a57692-a4eb-4651-aea1-82dade1b60d0 None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:40:45.277234 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a57692-a4eb-4651-aea1-82dade1b60d0 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:40:45.277368 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a57692-a4eb-4651-aea1-82dade1b60d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:40:45.307838 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-22a57692-a4eb-4651-aea1-82dade1b60d0 None None] MFA Rules not processed for user `593948925ca642c196cbdef04985374f`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:40:45.309240 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-22a57692-a4eb-4651-aea1-82dade1b60d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:40:45.378029 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1441/2912] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 12:40:45 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:40:45.382548 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68b34fa-c94b-4804-9c20-9dc8cd25c561 service neutron] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:40:45.392198 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68b34fa-c94b-4804-9c20-9dc8cd25c561 service neutron] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:40:45.397299 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3674dd-f475-42ba-96aa-0857c0f7cd6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:40:45.397896 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3674dd-f475-42ba-96aa-0857c0f7cd6a admin admin] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:40:45.398177 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3674dd-f475-42ba-96aa-0857c0f7cd6a admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:40:45.398445 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3674dd-f475-42ba-96aa-0857c0f7cd6a admin admin] PATH_INFO: `/v3/projects` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:40:45.399106 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3674dd-f475-42ba-96aa-0857c0f7cd6a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:40:45.401019 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3674dd-f475-42ba-96aa-0857c0f7cd6a admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:40:45.432062 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1472/2913] 10.222.0.15 () {60 vars in 1153 bytes} [Thu Sep 15 12:40:45 2022] POST /identity/v3/projects => generated 379 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 12:40:45.436315 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a57692-a4eb-4651-aea1-82dade1b60d0 None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:40:45.447705 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a57692-a4eb-4651-aea1-82dade1b60d0 None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:40:45.503788 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305a2188-84a4-47b9-aa7d-4f1cb15046ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:40:45.504421 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305a2188-84a4-47b9-aa7d-4f1cb15046ff admin admin] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:40:45.504696 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305a2188-84a4-47b9-aa7d-4f1cb15046ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:40:45.504994 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305a2188-84a4-47b9-aa7d-4f1cb15046ff admin admin] PATH_INFO: `/v3/users` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:40:45.505623 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305a2188-84a4-47b9-aa7d-4f1cb15046ff admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:40:45.507503 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305a2188-84a4-47b9-aa7d-4f1cb15046ff admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:40:45.527648 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1442/2914] 10.222.0.15 () {60 vars in 1144 bytes} [Thu Sep 15 12:40:45 2022] POST /identity/v3/users => generated 296 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Sep 15 12:40:45.531648 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3674dd-f475-42ba-96aa-0857c0f7cd6a admin admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:40:45.542476 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3674dd-f475-42ba-96aa-0857c0f7cd6a admin admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:40:45.547001 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89987e34-6d6f-47ea-827c-44758eab281b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:40:45.547595 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89987e34-6d6f-47ea-827c-44758eab281b admin admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:40:45.547873 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89987e34-6d6f-47ea-827c-44758eab281b admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:40:45.548145 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89987e34-6d6f-47ea-827c-44758eab281b admin admin] PATH_INFO: `/v3/roles` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:40:45.548798 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89987e34-6d6f-47ea-827c-44758eab281b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:40:45.550415 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89987e34-6d6f-47ea-827c-44758eab281b admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:40:45.560796 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1473/2915] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 12:40:45 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 12:40:45.564684 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305a2188-84a4-47b9-aa7d-4f1cb15046ff admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:40:45.574359 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305a2188-84a4-47b9-aa7d-4f1cb15046ff admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:40:45.578911 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ccee6d-1e26-4d30-9ed8-e769222ad17f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:40:45.579559 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ccee6d-1e26-4d30-9ed8-e769222ad17f admin admin] REQUEST_METHOD: `PUT` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:40:45.579840 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ccee6d-1e26-4d30-9ed8-e769222ad17f admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:40:45.580114 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ccee6d-1e26-4d30-9ed8-e769222ad17f admin admin] PATH_INFO: `/v3/projects/97a3cb76305d44dab351110ea2c879a8/users/e045b1c9c1664eaab197bc85f23391e2/roles/9c769b5c82884766b2c510b1f0f1364c` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:40:45.595382 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ccee6d-1e26-4d30-9ed8-e769222ad17f admin admin] RBAC: Authorizing `identity:create_grant(project_id=97a3cb76305d44dab351110ea2c879a8, user_id=e045b1c9c1664eaab197bc85f23391e2, role_id=9c769b5c82884766b2c510b1f0f1364c)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:40:45.598737 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ccee6d-1e26-4d30-9ed8-e769222ad17f admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:40:45.608154 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:40:45.615327 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1443/2916] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 15 12:40:45 2022] PUT /identity/v3/projects/97a3cb76305d44dab351110ea2c879a8/users/e045b1c9c1664eaab197bc85f23391e2/roles/9c769b5c82884766b2c510b1f0f1364c => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:40:45.618764 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89987e34-6d6f-47ea-827c-44758eab281b admin admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:40:45.629198 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89987e34-6d6f-47ea-827c-44758eab281b admin admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:40:45.652962 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:40:45.689387 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3217f8-52df-4a0d-a430-c539c6d67d04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:40:45.689927 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3217f8-52df-4a0d-a430-c539c6d67d04 admin admin] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:40:45.690723 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3217f8-52df-4a0d-a430-c539c6d67d04 admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:40:45.690994 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3217f8-52df-4a0d-a430-c539c6d67d04 admin admin] PATH_INFO: `/v3/roles` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:40:45.691628 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e3217f8-52df-4a0d-a430-c539c6d67d04 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:40:45.693237 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e3217f8-52df-4a0d-a430-c539c6d67d04 admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:40:45.702917 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1474/2917] 10.222.0.15 () {58 vars in 1122 bytes} [Thu Sep 15 12:40:45 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Sep 15 12:40:45.707714 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ccee6d-1e26-4d30-9ed8-e769222ad17f admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:40:45.718826 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ccee6d-1e26-4d30-9ed8-e769222ad17f admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:40:45.723497 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820f23d2-c5e5-430c-9a0a-58393480ec2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:40:45.724231 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820f23d2-c5e5-430c-9a0a-58393480ec2e admin admin] REQUEST_METHOD: `PUT` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:40:45.724630 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820f23d2-c5e5-430c-9a0a-58393480ec2e admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:40:45.725019 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-820f23d2-c5e5-430c-9a0a-58393480ec2e admin admin] PATH_INFO: `/v3/projects/97a3cb76305d44dab351110ea2c879a8/users/e045b1c9c1664eaab197bc85f23391e2/roles/9c769b5c82884766b2c510b1f0f1364c` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:40:45.727241 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-820f23d2-c5e5-430c-9a0a-58393480ec2e admin admin] RBAC: Authorizing `identity:create_grant(project_id=97a3cb76305d44dab351110ea2c879a8, user_id=e045b1c9c1664eaab197bc85f23391e2, role_id=9c769b5c82884766b2c510b1f0f1364c)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:40:45.729881 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-820f23d2-c5e5-430c-9a0a-58393480ec2e admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:40:45.739823 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1444/2918] 10.222.0.15 () {60 vars in 1483 bytes} [Thu Sep 15 12:40:45 2022] PUT /identity/v3/projects/97a3cb76305d44dab351110ea2c879a8/users/e045b1c9c1664eaab197bc85f23391e2/roles/9c769b5c82884766b2c510b1f0f1364c => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:40:50.134379 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23f6a35-7dc4-4740-9ace-e5181495228e None None] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:40:50.134859 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23f6a35-7dc4-4740-9ace-e5181495228e None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:40:50.135055 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23f6a35-7dc4-4740-9ace-e5181495228e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:40:50.166314 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.auth.core [None req-b23f6a35-7dc4-4740-9ace-e5181495228e None None] MFA Rules not processed for user `e045b1c9c1664eaab197bc85f23391e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81449) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:40:50.167754 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-b23f6a35-7dc4-4740-9ace-e5181495228e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:40:50.262868 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1475/2919] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 12:40:50 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:40:50.280855 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820f23d2-c5e5-430c-9a0a-58393480ec2e admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:40:50.282706 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-820f23d2-c5e5-430c-9a0a-58393480ec2e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:40:50.370893 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-820f23d2-c5e5-430c-9a0a-58393480ec2e admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:40:50.376290 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eada1cca-1c6c-426f-a1fd-6a436384c330 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eface89e261840a2abb4206be26455c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '540a32ef88764e8c8d9201ae22f6ddaf', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:40:50.376933 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eada1cca-1c6c-426f-a1fd-6a436384c330 service nova] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:40:50.377231 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eada1cca-1c6c-426f-a1fd-6a436384c330 service nova] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:40:50.377502 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eada1cca-1c6c-426f-a1fd-6a436384c330 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:40:50.435117 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eada1cca-1c6c-426f-a1fd-6a436384c330 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:40:50.436701 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eada1cca-1c6c-426f-a1fd-6a436384c330 service nova] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:40:50.445965 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1445/2920] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 12:40:50 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 12:41:00.259716 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=51 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:41:00.441725 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=61,keystone:INSERT=1 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:41:07.787076 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccbd417-8ca4-40ca-99df-7cae60998aa3 None None] REQUEST_METHOD: `POST` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:41:07.787545 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccbd417-8ca4-40ca-99df-7cae60998aa3 None None] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:41:07.787657 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccbd417-8ca4-40ca-99df-7cae60998aa3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:41:07.806568 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.auth.core [None req-fccbd417-8ca4-40ca-99df-7cae60998aa3 None None] MFA Rules not processed for user `593948925ca642c196cbdef04985374f`. Rule list: `[]` (Enabled: `True`). {{(pid=81449) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:41:07.807748 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-fccbd417-8ca4-40ca-99df-7cae60998aa3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:41:07.814377 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1476/2921] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 12:41:07 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:41:11.009316 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eada1cca-1c6c-426f-a1fd-6a436384c330 service nova] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:41:11.011112 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-eada1cca-1c6c-426f-a1fd-6a436384c330 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:41:11.032575 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eada1cca-1c6c-426f-a1fd-6a436384c330 service nova] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:41:11.041390 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840a64bc-8552-4ec0-ba20-b7daf9aceab1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eface89e261840a2abb4206be26455c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '540a32ef88764e8c8d9201ae22f6ddaf', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:41:11.042361 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840a64bc-8552-4ec0-ba20-b7daf9aceab1 service nova] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:41:11.042700 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840a64bc-8552-4ec0-ba20-b7daf9aceab1 service nova] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:41:11.043168 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840a64bc-8552-4ec0-ba20-b7daf9aceab1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:41:11.045605 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-840a64bc-8552-4ec0-ba20-b7daf9aceab1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:41:11.059741 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840a64bc-8552-4ec0-ba20-b7daf9aceab1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:41:11.075109 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840a64bc-8552-4ec0-ba20-b7daf9aceab1 service nova] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:41:11.075841 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1446/2922] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 12:41:11 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 67 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 12:41:17.801332 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=3 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:41:21.070179 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=6 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:41:24.139086 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fccbd417-8ca4-40ca-99df-7cae60998aa3 None None] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:41:24.194217 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fccbd417-8ca4-40ca-99df-7cae60998aa3 None None] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:41:24.197815 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f265d7-f91e-46ef-aa95-afa7d29709cc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfbb4d6809064f9cb9e06626ed7d6814', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '540a32ef88764e8c8d9201ae22f6ddaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:41:24.198400 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f265d7-f91e-46ef-aa95-afa7d29709cc service neutron] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:41:24.198681 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f265d7-f91e-46ef-aa95-afa7d29709cc service neutron] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:41:24.198952 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f265d7-f91e-46ef-aa95-afa7d29709cc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:41:24.208279 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f265d7-f91e-46ef-aa95-afa7d29709cc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:41:24.209725 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f265d7-f91e-46ef-aa95-afa7d29709cc service neutron] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:41:24.219237 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1477/2923] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 12:41:24 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 12:41:25.742563 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840a64bc-8552-4ec0-ba20-b7daf9aceab1 service nova] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:41:25.754098 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840a64bc-8552-4ec0-ba20-b7daf9aceab1 service nova] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:41:25.759522 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-313e4533-756d-4dcd-91ff-73969fbe447f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eface89e261840a2abb4206be26455c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '540a32ef88764e8c8d9201ae22f6ddaf', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:41:25.760141 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313e4533-756d-4dcd-91ff-73969fbe447f service nova] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:41:25.760420 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313e4533-756d-4dcd-91ff-73969fbe447f service nova] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:41:25.760686 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313e4533-756d-4dcd-91ff-73969fbe447f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:41:25.762762 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-313e4533-756d-4dcd-91ff-73969fbe447f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:41:25.775894 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313e4533-756d-4dcd-91ff-73969fbe447f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:41:25.777495 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-313e4533-756d-4dcd-91ff-73969fbe447f service nova] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:41:25.788035 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1447/2924] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 12:41:25 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 12:41:34.215093 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=26 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:41:35.783539 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=6 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:42:31.545126 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f265d7-f91e-46ef-aa95-afa7d29709cc service neutron] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:42:31.547098 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-50f265d7-f91e-46ef-aa95-afa7d29709cc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:42:31.693941 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f265d7-f91e-46ef-aa95-afa7d29709cc service neutron] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:42:31.699848 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27291163-f60d-464c-b62f-d52b329baef3 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f64a058bd5440cb5268f228aa6e392', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '540a32ef88764e8c8d9201ae22f6ddaf', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:42:31.700453 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27291163-f60d-464c-b62f-d52b329baef3 service placement] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:42:31.700728 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27291163-f60d-464c-b62f-d52b329baef3 service placement] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:42:31.701027 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27291163-f60d-464c-b62f-d52b329baef3 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:42:31.702720 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-27291163-f60d-464c-b62f-d52b329baef3 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:42:31.714809 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27291163-f60d-464c-b62f-d52b329baef3 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:42:31.716593 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27291163-f60d-464c-b62f-d52b329baef3 service placement] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:42:31.726654 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1478/2925] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 12:42:31 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 12:42:41.722276 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=63 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:43:27.829067 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e021d271-9303-43a5-82aa-141144e63461 None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:43:27.829562 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e021d271-9303-43a5-82aa-141144e63461 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:43:27.829680 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e021d271-9303-43a5-82aa-141144e63461 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:43:27.848209 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-e021d271-9303-43a5-82aa-141144e63461 None None] MFA Rules not processed for user `593948925ca642c196cbdef04985374f`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:43:27.849417 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-e021d271-9303-43a5-82aa-141144e63461 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:43:27.856014 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1448/2926] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 12:43:27 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:43:28.167713 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27291163-f60d-464c-b62f-d52b329baef3 service placement] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:43:28.177172 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27291163-f60d-464c-b62f-d52b329baef3 service placement] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:43:28.181600 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f648c3-4688-4ca3-84d3-768c321111b1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfbb4d6809064f9cb9e06626ed7d6814', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '540a32ef88764e8c8d9201ae22f6ddaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:43:28.182189 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f648c3-4688-4ca3-84d3-768c321111b1 service neutron] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:43:28.182463 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f648c3-4688-4ca3-84d3-768c321111b1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:43:28.182728 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f648c3-4688-4ca3-84d3-768c321111b1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:43:28.184448 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-38f648c3-4688-4ca3-84d3-768c321111b1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:43:28.194145 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f648c3-4688-4ca3-84d3-768c321111b1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:43:28.195600 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f648c3-4688-4ca3-84d3-768c321111b1 service neutron] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:43:28.203655 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1479/2927] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 12:43:28 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 12:43:28.641002 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e021d271-9303-43a5-82aa-141144e63461 None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:43:28.653561 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e021d271-9303-43a5-82aa-141144e63461 None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:43:28.658416 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec611423-b5c7-4c4d-bf85-7c65b6441f9a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f64a058bd5440cb5268f228aa6e392', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '540a32ef88764e8c8d9201ae22f6ddaf', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:43:28.659025 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec611423-b5c7-4c4d-bf85-7c65b6441f9a service placement] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:43:28.659301 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec611423-b5c7-4c4d-bf85-7c65b6441f9a service placement] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:43:28.659593 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec611423-b5c7-4c4d-bf85-7c65b6441f9a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:43:28.661303 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-ec611423-b5c7-4c4d-bf85-7c65b6441f9a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:43:28.673576 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec611423-b5c7-4c4d-bf85-7c65b6441f9a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:43:28.675206 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec611423-b5c7-4c4d-bf85-7c65b6441f9a service placement] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:43:28.685341 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1449/2928] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 12:43:28 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 12:43:38.199949 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=6 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:43:38.681080 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=9 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:43:40.623146 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f648c3-4688-4ca3-84d3-768c321111b1 service neutron] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:43:40.638883 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f648c3-4688-4ca3-84d3-768c321111b1 service neutron] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:43:40.644370 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9574a19-9673-47b7-a611-c82ab90853b5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eface89e261840a2abb4206be26455c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '540a32ef88764e8c8d9201ae22f6ddaf', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:43:40.645000 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9574a19-9673-47b7-a611-c82ab90853b5 service nova] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:43:40.645284 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9574a19-9673-47b7-a611-c82ab90853b5 service nova] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:43:40.645558 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9574a19-9673-47b7-a611-c82ab90853b5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:43:40.647296 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-a9574a19-9673-47b7-a611-c82ab90853b5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:43:40.659595 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9574a19-9673-47b7-a611-c82ab90853b5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:43:40.661193 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9574a19-9673-47b7-a611-c82ab90853b5 service nova] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:43:40.671968 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1480/2929] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 12:43:40 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 12:43:50.667295 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=6 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:45:30.408812 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a126c2-fb01-4bc3-9b2e-c3ba39a1cff8 None None] REQUEST_METHOD: `POST` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:30.409355 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a126c2-fb01-4bc3-9b2e-c3ba39a1cff8 None None] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:30.409355 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a126c2-fb01-4bc3-9b2e-c3ba39a1cff8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:30.428941 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.auth.core [None req-98a126c2-fb01-4bc3-9b2e-c3ba39a1cff8 None None] MFA Rules not processed for user `593948925ca642c196cbdef04985374f`. Rule list: `[]` (Enabled: `True`). {{(pid=81448) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Sep 15 12:45:30.430136 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-98a126c2-fb01-4bc3-9b2e-c3ba39a1cff8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:45:30.436845 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1450/2930] 10.222.0.15 () {58 vars in 958 bytes} [Thu Sep 15 12:45:30 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Sep 15 12:45:30.478152 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9574a19-9673-47b7-a611-c82ab90853b5 service nova] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:30.479576 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-a9574a19-9673-47b7-a611-c82ab90853b5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:45:30.559254 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9574a19-9673-47b7-a611-c82ab90853b5 service nova] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:30.562770 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886a4727-b754-4b46-ad32-48bd0c7765e7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c8191d3ccb874529b32f8d0026db629d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '540a32ef88764e8c8d9201ae22f6ddaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:30.563377 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886a4727-b754-4b46-ad32-48bd0c7765e7 service cinder] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:30.563655 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886a4727-b754-4b46-ad32-48bd0c7765e7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:30.563926 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886a4727-b754-4b46-ad32-48bd0c7765e7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:30.565648 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-886a4727-b754-4b46-ad32-48bd0c7765e7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:45:30.596447 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886a4727-b754-4b46-ad32-48bd0c7765e7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:45:30.597911 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886a4727-b754-4b46-ad32-48bd0c7765e7 service cinder] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:45:30.619364 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1481/2931] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 12:45:30 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 12:45:31.344622 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a126c2-fb01-4bc3-9b2e-c3ba39a1cff8 None None] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:31.363655 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a126c2-fb01-4bc3-9b2e-c3ba39a1cff8 None None] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:31.367379 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9cdbf5-b394-47bd-a58e-b96a799cface service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfbb4d6809064f9cb9e06626ed7d6814', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '540a32ef88764e8c8d9201ae22f6ddaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:31.368063 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9cdbf5-b394-47bd-a58e-b96a799cface service neutron] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:31.368414 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9cdbf5-b394-47bd-a58e-b96a799cface service neutron] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:31.368782 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9cdbf5-b394-47bd-a58e-b96a799cface service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:31.370606 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-bb9cdbf5-b394-47bd-a58e-b96a799cface service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:45:31.385686 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9cdbf5-b394-47bd-a58e-b96a799cface service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:45:31.387101 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9cdbf5-b394-47bd-a58e-b96a799cface service neutron] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:45:31.398923 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1451/2932] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 12:45:31 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 12:45:32.528595 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886a4727-b754-4b46-ad32-48bd0c7765e7 service cinder] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:32.530688 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-886a4727-b754-4b46-ad32-48bd0c7765e7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:45:32.558343 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886a4727-b754-4b46-ad32-48bd0c7765e7 service cinder] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:32.562423 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78daafa6-0233-4984-a28a-cbd5eb70127e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfbb4d6809064f9cb9e06626ed7d6814', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '540a32ef88764e8c8d9201ae22f6ddaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:32.563730 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78daafa6-0233-4984-a28a-cbd5eb70127e service neutron] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:32.564435 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78daafa6-0233-4984-a28a-cbd5eb70127e service neutron] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:32.565019 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78daafa6-0233-4984-a28a-cbd5eb70127e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:32.566993 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-78daafa6-0233-4984-a28a-cbd5eb70127e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:45:32.579210 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78daafa6-0233-4984-a28a-cbd5eb70127e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:45:32.580652 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78daafa6-0233-4984-a28a-cbd5eb70127e service neutron] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:45:32.590819 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1482/2933] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 12:45:32 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 12:45:41.393368 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=13 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:45:42.586597 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=58 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:45:50.060318 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9cdbf5-b394-47bd-a58e-b96a799cface service neutron] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:45:50.069163 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9cdbf5-b394-47bd-a58e-b96a799cface service neutron] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:45:50.072776 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea4f650-27a6-4040-8a3a-a2b8f68456e0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfbb4d6809064f9cb9e06626ed7d6814', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '540a32ef88764e8c8d9201ae22f6ddaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:45:50.073369 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea4f650-27a6-4040-8a3a-a2b8f68456e0 service neutron] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:45:50.073648 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea4f650-27a6-4040-8a3a-a2b8f68456e0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:45:50.073914 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea4f650-27a6-4040-8a3a-a2b8f68456e0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:45:50.082006 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea4f650-27a6-4040-8a3a-a2b8f68456e0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:45:50.083401 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea4f650-27a6-4040-8a3a-a2b8f68456e0 service neutron] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:45:50.092190 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1452/2934] 10.222.0.15 () {60 vars in 1396 bytes} [Thu Sep 15 12:45:50 2022] GET /identity/v3/auth/tokens?allow_expired=1 => generated 3254 bytes in 32 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 12:46:00.088360 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=6 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:47:30.893178 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78daafa6-0233-4984-a28a-cbd5eb70127e service neutron] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:47:30.906994 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78daafa6-0233-4984-a28a-cbd5eb70127e service neutron] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:47:30.912938 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8368995b-45ff-4f58-adc4-0854adcde179 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eface89e261840a2abb4206be26455c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '540a32ef88764e8c8d9201ae22f6ddaf', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:47:30.913645 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8368995b-45ff-4f58-adc4-0854adcde179 service nova] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:47:30.913993 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8368995b-45ff-4f58-adc4-0854adcde179 service nova] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:47:30.914335 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8368995b-45ff-4f58-adc4-0854adcde179 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:47:30.923889 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8368995b-45ff-4f58-adc4-0854adcde179 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:47:30.925638 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8368995b-45ff-4f58-adc4-0854adcde179 service nova] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:47:30.937395 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1483/2935] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 12:47:30 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 12:47:33.319163 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea4f650-27a6-4040-8a3a-a2b8f68456e0 service neutron] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:47:33.331800 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea4f650-27a6-4040-8a3a-a2b8f68456e0 service neutron] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:47:33.338313 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51a2f1d-9bd8-40d3-bebd-2dfd63064ec2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'eface89e261840a2abb4206be26455c7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '540a32ef88764e8c8d9201ae22f6ddaf', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'reader', 'member', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:47:33.338928 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51a2f1d-9bd8-40d3-bebd-2dfd63064ec2 service nova] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:47:33.339205 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51a2f1d-9bd8-40d3-bebd-2dfd63064ec2 service nova] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:47:33.339473 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51a2f1d-9bd8-40d3-bebd-2dfd63064ec2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:47:33.349615 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51a2f1d-9bd8-40d3-bebd-2dfd63064ec2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:47:33.351391 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51a2f1d-9bd8-40d3-bebd-2dfd63064ec2 service nova] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:47:33.362524 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1453/2936] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 12:47:33 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 12:47:33.728310 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8368995b-45ff-4f58-adc4-0854adcde179 service nova] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:47:33.729707 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-8368995b-45ff-4f58-adc4-0854adcde179 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:47:33.744414 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8368995b-45ff-4f58-adc4-0854adcde179 service nova] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:47:33.749298 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb43384-d678-4ca6-ab2f-1bfc98e4e53e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f64a058bd5440cb5268f228aa6e392', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '540a32ef88764e8c8d9201ae22f6ddaf', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:47:33.749872 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb43384-d678-4ca6-ab2f-1bfc98e4e53e service placement] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:47:33.750169 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb43384-d678-4ca6-ab2f-1bfc98e4e53e service placement] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:47:33.750420 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb43384-d678-4ca6-ab2f-1bfc98e4e53e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:47:33.759651 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb43384-d678-4ca6-ab2f-1bfc98e4e53e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:47:33.761325 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb43384-d678-4ca6-ab2f-1bfc98e4e53e service placement] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:47:33.771338 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1484/2937] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 12:47:33 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 12:47:43.155955 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51a2f1d-9bd8-40d3-bebd-2dfd63064ec2 service nova] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:47:43.165823 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51a2f1d-9bd8-40d3-bebd-2dfd63064ec2 service nova] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:47:43.169325 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93fad4d1-8544-4f49-b9ba-9910dbd8c5d2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bfbb4d6809064f9cb9e06626ed7d6814', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '540a32ef88764e8c8d9201ae22f6ddaf', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:47:43.169920 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fad4d1-8544-4f49-b9ba-9910dbd8c5d2 service neutron] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:47:43.170195 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fad4d1-8544-4f49-b9ba-9910dbd8c5d2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:47:43.170464 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fad4d1-8544-4f49-b9ba-9910dbd8c5d2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:47:43.240676 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93fad4d1-8544-4f49-b9ba-9910dbd8c5d2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:47:43.242111 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93fad4d1-8544-4f49-b9ba-9910dbd8c5d2 service neutron] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:47:43.260194 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1454/2938] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 12:47:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 12:47:43.767053 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=12 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:47:46.262983 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb43384-d678-4ca6-ab2f-1bfc98e4e53e service placement] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:47:46.274507 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb43384-d678-4ca6-ab2f-1bfc98e4e53e service placement] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:47:46.279898 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05702319-5c65-480e-832f-b2c559e0e94b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f64a058bd5440cb5268f228aa6e392', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '540a32ef88764e8c8d9201ae22f6ddaf', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:47:46.280627 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05702319-5c65-480e-832f-b2c559e0e94b service placement] REQUEST_METHOD: `GET` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:47:46.281033 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05702319-5c65-480e-832f-b2c559e0e94b service placement] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:47:46.281408 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05702319-5c65-480e-832f-b2c559e0e94b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:47:46.283222 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-05702319-5c65-480e-832f-b2c559e0e94b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:47:46.299172 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05702319-5c65-480e-832f-b2c559e0e94b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:47:46.301468 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05702319-5c65-480e-832f-b2c559e0e94b service placement] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:47:46.326018 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1485/2939] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 12:47:46 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 12:47:49.600646 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93fad4d1-8544-4f49-b9ba-9910dbd8c5d2 service neutron] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:47:49.611132 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93fad4d1-8544-4f49-b9ba-9910dbd8c5d2 service neutron] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:47:49.615983 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346912d3-2e84-41d6-a112-c8ae452affab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:47:49.616617 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346912d3-2e84-41d6-a112-c8ae452affab admin admin] REQUEST_METHOD: `DELETE` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:47:49.616938 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346912d3-2e84-41d6-a112-c8ae452affab admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:47:49.617213 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346912d3-2e84-41d6-a112-c8ae452affab admin admin] PATH_INFO: `/v3/users/593948925ca642c196cbdef04985374f` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:47:49.618344 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346912d3-2e84-41d6-a112-c8ae452affab admin admin] RBAC: Authorizing `identity:delete_user(user_id=593948925ca642c196cbdef04985374f)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:47:49.620246 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346912d3-2e84-41d6-a112-c8ae452affab admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:47:49.667249 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-346912d3-2e84-41d6-a112-c8ae452affab admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '593948925ca642c196cbdef04985374f'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:47:49.678956 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-346912d3-2e84-41d6-a112-c8ae452affab admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '593948925ca642c196cbdef04985374f'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:47:49.686385 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-346912d3-2e84-41d6-a112-c8ae452affab admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '593948925ca642c196cbdef04985374f'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:47:49.687171 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-346912d3-2e84-41d6-a112-c8ae452affab admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '593948925ca642c196cbdef04985374f'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:47:49.694252 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-346912d3-2e84-41d6-a112-c8ae452affab admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '593948925ca642c196cbdef04985374f'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:47:49.700065 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1455/2940] 10.222.0.15 () {58 vars in 1224 bytes} [Thu Sep 15 12:47:49 2022] DELETE /identity/v3/users/593948925ca642c196cbdef04985374f => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:47:49.868167 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05702319-5c65-480e-832f-b2c559e0e94b service placement] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:47:49.878544 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05702319-5c65-480e-832f-b2c559e0e94b service placement] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:47:49.933701 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45893013-09ec-4233-9bbe-c54ce86bf7dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:47:49.934333 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45893013-09ec-4233-9bbe-c54ce86bf7dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:47:49.934611 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45893013-09ec-4233-9bbe-c54ce86bf7dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:47:49.934880 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45893013-09ec-4233-9bbe-c54ce86bf7dd admin admin] PATH_INFO: `/v3/projects/923986fe38f542549682ddff9eb63ff1` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:47:49.935968 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45893013-09ec-4233-9bbe-c54ce86bf7dd admin admin] RBAC: Authorizing `identity:delete_project(project_id=923986fe38f542549682ddff9eb63ff1)` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:47:49.937991 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45893013-09ec-4233-9bbe-c54ce86bf7dd admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:47:49.983896 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-45893013-09ec-4233-9bbe-c54ce86bf7dd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '923986fe38f542549682ddff9eb63ff1', 'request_id': 'req-45893013-09ec-4233-9bbe-c54ce86bf7dd'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:47:49.990791 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-45893013-09ec-4233-9bbe-c54ce86bf7dd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '923986fe38f542549682ddff9eb63ff1', 'request_id': 'req-45893013-09ec-4233-9bbe-c54ce86bf7dd'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:47:49.996380 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-45893013-09ec-4233-9bbe-c54ce86bf7dd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '923986fe38f542549682ddff9eb63ff1', 'request_id': 'req-45893013-09ec-4233-9bbe-c54ce86bf7dd'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:47:50.019182 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-45893013-09ec-4233-9bbe-c54ce86bf7dd admin admin] The token cache is being invalidate because project 923986fe38f542549682ddff9eb63ff1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81449) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 12:47:50.019476 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-45893013-09ec-4233-9bbe-c54ce86bf7dd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:47:50.020717 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-45893013-09ec-4233-9bbe-c54ce86bf7dd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:47:50.022196 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1486/2941] 10.222.0.15 () {58 vars in 1233 bytes} [Thu Sep 15 12:47:49 2022] DELETE /identity/v3/projects/923986fe38f542549682ddff9eb63ff1 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:47:50.026098 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346912d3-2e84-41d6-a112-c8ae452affab admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:47:50.027555 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-346912d3-2e84-41d6-a112-c8ae452affab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:47:50.040916 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346912d3-2e84-41d6-a112-c8ae452affab admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:47:50.074298 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:47:50.101062 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d342fc1d-e7bb-4a8d-8d74-afc8d4437739 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:47:50.101710 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d342fc1d-e7bb-4a8d-8d74-afc8d4437739 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:47:50.102002 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d342fc1d-e7bb-4a8d-8d74-afc8d4437739 admin admin] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:47:50.102263 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d342fc1d-e7bb-4a8d-8d74-afc8d4437739 admin admin] PATH_INFO: `/v3/users/e045b1c9c1664eaab197bc85f23391e2` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:47:50.103399 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d342fc1d-e7bb-4a8d-8d74-afc8d4437739 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e045b1c9c1664eaab197bc85f23391e2)` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:47:50.105351 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d342fc1d-e7bb-4a8d-8d74-afc8d4437739 admin admin] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:47:50.153618 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-d342fc1d-e7bb-4a8d-8d74-afc8d4437739 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e045b1c9c1664eaab197bc85f23391e2'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:47:50.166709 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-d342fc1d-e7bb-4a8d-8d74-afc8d4437739 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e045b1c9c1664eaab197bc85f23391e2'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:47:50.174292 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-d342fc1d-e7bb-4a8d-8d74-afc8d4437739 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e045b1c9c1664eaab197bc85f23391e2'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:47:50.175001 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-d342fc1d-e7bb-4a8d-8d74-afc8d4437739 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e045b1c9c1664eaab197bc85f23391e2'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:47:50.182429 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.notifications [None req-d342fc1d-e7bb-4a8d-8d74-afc8d4437739 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e045b1c9c1664eaab197bc85f23391e2'} {{(pid=81448) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:47:50.188193 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1456/2942] 10.222.0.15 () {58 vars in 1224 bytes} [Thu Sep 15 12:47:50 2022] DELETE /identity/v3/users/e045b1c9c1664eaab197bc85f23391e2 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:47:50.361069 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45893013-09ec-4233-9bbe-c54ce86bf7dd admin admin] Authenticating user token {{(pid=81449) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:47:50.362555 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.fernet_utils [None req-45893013-09ec-4233-9bbe-c54ce86bf7dd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81449) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:47:50.376935 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45893013-09ec-4233-9bbe-c54ce86bf7dd admin admin] Validating token access rules against request {{(pid=81449) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:47:50.434223 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a397e93-7c15-4aae-8671-1c4a4d91b77a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c66eff9fe7124011868989f16846a501', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '69bea7b802b04b689af0d31b020f8554', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81449) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:47:50.434860 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a397e93-7c15-4aae-8671-1c4a4d91b77a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:47:50.435141 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a397e93-7c15-4aae-8671-1c4a4d91b77a admin admin] SCRIPT_NAME: `/identity` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:47:50.435415 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a397e93-7c15-4aae-8671-1c4a4d91b77a admin admin] PATH_INFO: `/v3/projects/97a3cb76305d44dab351110ea2c879a8` {{(pid=81449) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:47:50.436514 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a397e93-7c15-4aae-8671-1c4a4d91b77a admin admin] RBAC: Authorizing `identity:delete_project(project_id=97a3cb76305d44dab351110ea2c879a8)` {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:47:50.438561 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a397e93-7c15-4aae-8671-1c4a4d91b77a admin admin] RBAC: Authorization granted {{(pid=81449) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:47:50.447133 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:47:50.485552 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-2a397e93-7c15-4aae-8671-1c4a4d91b77a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '97a3cb76305d44dab351110ea2c879a8', 'request_id': 'req-2a397e93-7c15-4aae-8671-1c4a4d91b77a'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:47:50.492979 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-2a397e93-7c15-4aae-8671-1c4a4d91b77a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '97a3cb76305d44dab351110ea2c879a8', 'request_id': 'req-2a397e93-7c15-4aae-8671-1c4a4d91b77a'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:47:50.498409 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-2a397e93-7c15-4aae-8671-1c4a4d91b77a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '97a3cb76305d44dab351110ea2c879a8', 'request_id': 'req-2a397e93-7c15-4aae-8671-1c4a4d91b77a'} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:47:50.509638 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-2a397e93-7c15-4aae-8671-1c4a4d91b77a admin admin] The token cache is being invalidate because project 97a3cb76305d44dab351110ea2c879a8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81449) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Sep 15 12:47:50.509918 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-2a397e93-7c15-4aae-8671-1c4a4d91b77a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:47:50.510553 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG keystone.notifications [None req-2a397e93-7c15-4aae-8671-1c4a4d91b77a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81449) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Sep 15 12:47:50.511997 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: [pid: 81449|app: 0|req: 1487/2943] 10.222.0.15 () {58 vars in 1233 bytes} [Thu Sep 15 12:47:50 2022] DELETE /identity/v3/projects/97a3cb76305d44dab351110ea2c879a8 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Sep 15 12:48:00.184724 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=37,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:48:00.506696 dv-cinder-850836-7-iscsi devstack@keystone.service[81449]: DEBUG dbcounter [-] [81449] Writing DB stats keystone:SELECT=21,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81449) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Sep 15 12:52:42.775302 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d342fc1d-e7bb-4a8d-8d74-afc8d4437739 admin admin] Authenticating user token {{(pid=81448) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Sep 15 12:52:42.776681 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-d342fc1d-e7bb-4a8d-8d74-afc8d4437739 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:52:42.910351 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d342fc1d-e7bb-4a8d-8d74-afc8d4437739 admin admin] Validating token access rules against request {{(pid=81448) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Sep 15 12:52:42.967264 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57f80a8-cd3a-47c7-99dc-d153b686981e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '51f64a058bd5440cb5268f228aa6e392', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '540a32ef88764e8c8d9201ae22f6ddaf', 'project_domain_id': 'default', 'roles': ['service', 'admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81448) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Sep 15 12:52:42.967869 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57f80a8-cd3a-47c7-99dc-d153b686981e service placement] REQUEST_METHOD: `GET` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Sep 15 12:52:42.968161 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57f80a8-cd3a-47c7-99dc-d153b686981e service placement] SCRIPT_NAME: `/identity` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Sep 15 12:52:42.968411 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57f80a8-cd3a-47c7-99dc-d153b686981e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81448) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Sep 15 12:52:42.970126 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.fernet_utils [None req-a57f80a8-cd3a-47c7-99dc-d153b686981e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81448) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Sep 15 12:52:42.981932 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57f80a8-cd3a-47c7-99dc-d153b686981e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Sep 15 12:52:42.983481 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57f80a8-cd3a-47c7-99dc-d153b686981e service placement] RBAC: Authorization granted {{(pid=81448) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Sep 15 12:52:42.993193 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: [pid: 81448|app: 0|req: 1457/2944] 10.222.0.15 () {60 vars in 1365 bytes} [Thu Sep 15 12:52:42 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 218 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Sep 15 12:52:52.988979 dv-cinder-850836-7-iscsi devstack@keystone.service[81448]: DEBUG dbcounter [-] [81448] Writing DB stats keystone:SELECT=89 {{(pid=81448) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}